CN103325120A - Rapid self-adaption binocular vision stereo matching method capable of supporting weight - Google Patents

Abstract

The invention discloses a rapid self-adaption binocular vision stereo matching method capable of supporting a weight. The method includes the following steps of reading an existing binocular vision image pair to be matched, calculating the matching cost, aggregating the matching cost in a weighted mode, calculating initial parallax, correcting the initial parallax, obtaining a final parallax matrix, generating a parallax graph and outputting a result. The rapid self-adaption binocular vision stereo matching method capable of supporting the weight can be used for the technical field of stereo display, and the stereo matching effect is improved.

Description

A kind of quick self-adapted support-weight binocular vision solid matching method

Technical field

The present invention relates to the image display technology field, relate in particular to a kind of quick self-adapted support-weight binocular vision solid matching method.

Background technology

Vision is an important means in the human knowledge world, the perception world, in the human cognition to external world approximately 75% information be to obtain by vision system.From traditional black-and-white photograph, black and white television set high-resolution color digital photo, digital TV in high resolution till now, human more and more higher to the visual experience requirement.Although traditional two-dimensional video can provide the two dimensional surface information of high definition, the human lives is in three-dimensional world, and the two dimensional surface video can't be given people's a kind of " on the spot in person " visual experience all the time.The break traditions limitation in two-dimensional video " the simple eye world of seeing " of binocular stereo vision, utilize the computer simulation mankind's vision system, obtain the three-dimensional information of scene by two width of cloth in the scene or several two dimension views, the mankind just can experience real three-dimensional world by the stereo display video.Binocular stereo vision is an important research field of computer vision, comprise four steps: Image Acquisition, camera calibration, Stereo matching and three-dimensional reconstruction, wherein Stereo matching is gordian technique, and the precision of Stereo matching directly has influence on the effect of three-dimensional reconstruction.

Although still there are many problems in actual applications in existing a large amount of solid matching method at present.Stereo matching can be divided into two classes according to the difference of optimization method: overall solid matching method and local Stereo matching side.Overall situation solid matching method matching precision is high, but computation structure is complicated, is not easy to hardware and realizes, the sectional perspective matching process is simple in structure, be easy to hardware and realize, but matching precision is also relatively low.Since Yoon proposition self-adaptation support-weight method, the Stereo matching performance of partial approach improves greatly, even surpassed some global approach, but still there is a major issue in Yoon self-adaptation support-weight method: computing velocity is slow, and computing time is longer than other local algorithm.Therefore, invent the quick self-adapted support-weight method tool that a kind of computing velocity is fast, matching performance is high and be of great significance, be conducive to Stereo Matching Technology is applied in the practical problems.

Summary of the invention

Technical matters to be solved by this invention is to overcome the deficiency of Yoon self-adaptation support-weight method, the quick self-adapted support-weight method of a kind of extension-based order conversion is proposed, take the right left view of binocular stereo image as image to be matched, take right view as matching image, for each pixel to be matched in the left view finds corresponding match point in right view, namely try to achieve the disparity map of left view.

The present invention is for solving its technical matters, and the technology that adopts comprises following steps:

S1, read existing binocular image to be matched to I _lAnd I _r, size and the Color Channel information of acquisition image, wherein, I _lThe expression left view is image to be matched, I _rThe expression right view is matching image;

S2, calculating I _l, I _rCoupling cost between middle pixel comprises:

S21, treat matching image I _l, determine square support window N, calculate and support center pixel and the gray scale difference diff (p that supports pixel in the window, q), wherein, diff (p, q)=and I (p)-I (q), I (p) and I (q) are respectively central pixel point p and support the gray-scale value of pixel q;

S22, according to S21 gained gray scale difference diff (p, q), with each pixel definition is in 5 grades in the square support window N, five grades are as follows:

Wherein, s and t are the threshold values that rule of thumb arranges, and satisfy this principle that affects that reduces as far as possible picture noise;

S23, statistics draw initial similarity measure value S _d:

Make fuz represent the numerical matrix that obtains after the order conversion, matrix size is identical with square support window N size, $\mathrm{fuz}=\left\{\begin{array}{cc}-2& \mathrm{diff}(p,q)<-s\\ -1& -s\&le;\mathrm{diff}(p,q)<-t\\ 0& -t\&le;\mathrm{diff}(p,q)\&le;t\\ 1& t<\mathrm{diff}(p,q)\&le;s\\ 2& s<\mathrm{diff}(p,q)\end{array}\right.,$ Respectively to I _lIn pixel to be matched and I _rMiddle candidate matches pixel carries out the order conversion, can obtain two order transformation matrix fuz _lAnd fuz _r, statistics order transformation matrix fuz _lWith order transformation matrix fuz _rCorrespondence position has the number of same levels in square support window, obtains initial similarity measure value S _d, wherein, $S_d={\mathrm{\&Sigma;}}_{q\&Element;N}m,$ $m=\left\{\begin{array}{cc}1& \mathrm{if}{\mathrm{fuz}}_1={\mathrm{fuz}}_r\\ 0& \mathrm{otherwise}\end{array}\right.,$ M represents order transformation matrix fuz _lAnd fuz _rWhether have same levels at correspondence position, if identical, m=1 then, otherwise m=0;

Initial similarity measure value corresponding to each parallax value d ∈ D in the square statistic window M of n * n in S24, the statistics image to be matched centered by the pixel to be matched obtains the coupling cost between the candidate matches pixel in pixel to be matched and the matching image according to ERT similarity measurement function

Wherein, d represents pixel to be matched and candidate matches pixel parallax in the horizontal direction, D={d _Min... d _Max, $C_{\mathrm{ERT}}(q,{\stackrel{\&OverBar;}{q}}_d)=\underset{n\&times;n}{\mathrm{\&Sigma;}}{S}_d\left(q\right);$

S3, weighting polymerization coupling cost comprise:

S31, calculating support-weight w (p, q): utilize color similarity and how much proximities, calculate the support-weight w (p, q) that the interior support of Matching supporting window pixel q treats matched pixel p, w (p, q)=f _s(Δ c _Pq) f _p(Δ g _Pq),

Wherein, f _s(Δ c _Pq) represent by the definite cluster intensity of color similarity, f _p(Δ g _Pq) represent by how much definite cluster intensity of proximity, Δ c _PqRepresent two pixel color c _pAnd c _qAt the Euclidean distance of RGB color space, c _p=[R _p, G _p, B _p], c _q=[R _q, G _q, B _q], ${\mathrm{\&Delta;c}}_{\mathrm{pq}}=\sqrt{{(R_p-R_q)}^2+{(G_p-G_q)}^2+{(B_p-B_q)}^2},$ Δ g _PqThe Euclidean distance on the locus of expression center pixel and support pixel, establishing pixel p is p (x, y) at the coordinate of image area, pixel q is q (x', y') at the coordinate of image area, then γ _c, γ _pBe user-specified parameters, be used for respectively the impact on the support-weight size of adjustable colors similarity and how much proximities;

S32, according to S24 gained coupling cost

With S31 gained support-weight w (p, q), weighting polymerization coupling cost obtains: $E(p,{\stackrel{\&OverBar;}{p}}_d)=\frac{{\mathrm{\&Sigma;}}_{q\&Element;N_P,\stackrel{\&OverBar;}{q}\&Element;N_{{\stackrel{\&OverBar;}{p}}_d}}w(p,q)c_{\mathrm{ERT}}(q,{\stackrel{\&OverBar;}{q}}_d)}{{\mathrm{\&Sigma;}}_{q\&Element;N_P,\stackrel{\&OverBar;}{q}\&Element;N_{{\stackrel{\&OverBar;}{p}}_d}}w(p,q)},$ Wherein,

The matched pixel point that represents respectively image pixel p to be matched, q correspondence in matching image when parallax is d, N _pSupport window size in the expression reference picture,

Corresponding support window size in the expression target image, and

S4, calculating initial parallax: gained weighting polymerization coupling cost among the S3 is adopted suboptimization method WTA(Winner-Take-All, the victor is a king), draw the maximum weighted polymerization result, the parallax value that the maximum weighted polymerization result is corresponding is the initial parallax d of this pixel _p, the initial parallax result of each pixel is saved in the initial parallax matrix, and the initial parallax matrix is: $d_p=\arg \underset{d\&Element;D}{\max }E(p,{\stackrel{\&OverBar;}{p}}_d);$

S5, S4 gained initial parallax is proofreaied and correct, is obtained final parallax matrix, comprising:

S51, definite correcting window N centered by pixel p to be corrected _c, be that each pixel is distributed a suitable support-weight w adaptively in the correction window according to color similarity and how much proximities _c,

The initial parallax of all pixels distributes in S52, the observation correction window, the number of times that statistical parallax d ∈ D occurs in correction window, and the number of times that in correction window, occurs of each parallax value of polymerization d and corresponding weights, the parallax that maximum polymerization result is corresponding then is the final parallax d of pixel to be corrected _{P_final}, and the result is saved in final parallax matrix, wherein $d_{p\_\mathrm{final}}=\arg \underset{d\&Element;D}{\max }\{{\mathrm{\&Sigma;}}_{q\&Element;N_c}{w}_c(p,q)\&times;k\},$ $k=\left\{\begin{array}{cc}1& \mathrm{if}{d}_p\left(q\right)=d\\ 0& \mathrm{otherwise}\end{array}\right.,$ K represents whether the initial parallax of pixel equals to treat statistical parallax d in the correction window, if equate, and k=1 then, otherwise k=0;

S6, generation disparity map, Output rusults: with the final parallax value d of S5 gained _{P_final}Be mapped to corresponding gray space [0,255], the mapping ratio is t, obtains representing the gray level image of parallax information.

Further, s＜t described in the S22.

Further, pixel should be as far as possible from same depth in the described correction window of S51, and the square correction window of self-adaptation support-weight satisfies this condition.

The present invention is by the quick self-adapted support-weight method of a kind of extension-based order conversion, take the right left view of binocular stereo image as image to be matched, take right view as matching image, for each pixel to be matched in the left view finds corresponding match point in right view, namely try to achieve the disparity map of left view, with respect to other computing method, computing velocity is fast, matching performance is high, more is conducive to Stereo Matching Technology is applied in the practical problems.

Description of drawings

Fig. 1 is method step schematic diagram of the present invention.

Embodiment

Below in conjunction with embodiment the present invention is described in further detail.

A kind of quick self-adapted support-weight binocular vision solid matching method its objective is and tries to achieve fast the right high precision dense disparity map of image to be matched.The Teddy standard testing image that the present embodiment provides take the Middlebury test platform is to as experimental subjects, wherein take left view as image to be matched, take right view as matching image, for each pixel to be matched in the left view finds corresponding match point in right view.In order to find the solution this problem, shown in Figure 1 according to flow process, need take following steps:

S1: read binocular image to be matched pair.The Teddy standard testing image that input Middlebury test platform provides pair, wherein left view I _l, right view I _rRead image to be matched to comprising the information such as image size and Color Channel.

S2: use expansion order transforming function transformation function, calculate I _lAnd I _rCoupling cost between pixel.

At first determine pixel p to be matched, determine square support window size N=25, calculate center pixel p and the gray scale difference diff (p that supports pixel q in the square support window, q)=I (p)-I (q), I (p) wherein, I (q) represents respectively pixel p, the gray-scale value of q.

Then according to the size of diff (p, q) value, will support each pixel definition to 5 grade in the window.

Left view and right view are carried out the order conversion, obtain two order transformation matrix fuz _l(p),

Order transformation matrix fuz in the square support window in the view of the statistics left and right sides _l(p) and

The number that has same levels at correspondence position obtains pixel p to be matched and candidate matches pixel

Between initial similarity measure value S _d

$S_d(q,{\stackrel{\&OverBar;}{q}}_d)={\mathrm{\&Sigma;}}_{q\&Element;N}m$

$m=\left\{\begin{array}{cc}1& \mathrm{if}({\mathrm{fuz}}_l\left(p\right)={\mathrm{fuz}}_r\left({\stackrel{\&OverBar;}{p}}_d\right))\\ 0& \mathrm{otherwise}\end{array}\right.$

Add up at last initial similarity measure value corresponding to each parallax value d ∈ D in 3 * 3 square statistical windows take pixel p to be matched as window center, obtain the coupling cost between pixel to be matched and candidate matches pixel

Wherein, initialization parallax collection D={0,1,2,3 ... 56,57,58,59}.

S3: power polymerization coupling cost:

At first, utilize color similarity and how much proximities, calculate and support in the Matching supporting window that pixel q treats the support-weight w (p, q) of matched pixel p:

$w(p,q)=f_s\left({\mathrm{\&Delta;c}}_{\mathrm{pq}}\right)\&CenterDot;f_p\left({\mathrm{\&Delta;g}}_{\mathrm{pq}}\right)=\exp (-(\frac{{\mathrm{\&Delta;c}}_{\mathrm{pq}}}{19}+\frac{{\mathrm{\&Delta;g}}_{\mathrm{pq}}}{12.5}))$

${\mathrm{\&Delta;c}}_{\mathrm{pq}}=\sqrt{{(R_p-R_q)}^2+{(G_p-G_q)}^2+{(B_p-B_q)}^2}$

${\mathrm{\&Delta;g}}_{\mathrm{pq}}=\sqrt{{(p(x,y)-q(x^{\&prime;},y^{\&prime;}))}^2}$

Then, use square support-weight window, polymerization coupling cost and corresponding support-weight, the size of support window is $N=N_p=N_{{\stackrel{\&OverBar;}{p}}_d}=25.$

$E(p,{\stackrel{\&OverBar;}{p}}_d)=\frac{{\mathrm{\&Sigma;}}_{q\&Element;N_P,\stackrel{\&OverBar;}{q}\&Element;N_{{\stackrel{\&OverBar;}{p}}_d}}w(p,q)c_{\mathrm{ERT}}(q,{\stackrel{\&OverBar;}{q}}_d)}{{\mathrm{\&Sigma;}}_{q\&Element;N_P,\stackrel{\&OverBar;}{q}\&Element;N_{{\stackrel{\&OverBar;}{p}}_d}}w(p,q)}$

S4: calculate initial parallax from polymerization result.Adopt suboptimization method WTA, have parallax value corresponding to maximum weighted result and then be the initial parallax d of this pixel _p, and the result is saved in the initial parallax matrix.

$d_p=\arg \underset{d\&Element;D}{\max }E(p,{\stackrel{\&OverBar;}{p}}_d)$

S5: initial parallax is proofreaied and correct

At first, determine correction window N centered by pixel p to be corrected _c=21, and according to color similarity and how much proximities, correction pixels p treats the support-weight w of correction pixels q in the calculation correction window _c(p, q):

$w_c(p,q)=\exp (-(\frac{{\mathrm{\&Delta;c}}_{\mathrm{pq}}}{19}+\frac{{\mathrm{\&Delta;g}}_{\mathrm{pq}}}{10.5}))$

Then the initial parallax of observing all pixels in the correction window distributes, the number of times that statistical parallax d ∈ D occurs in correction window, and the number of times that in calibration window, occurs of each parallax value of polymerization d and corresponding weights, the parallax that maximum polymerization result is corresponding then is the final parallax d of pixel to be corrected _{P_final}, and final parallax result is saved in final parallax matrix.

$d_{p\_\mathrm{final}}=\arg \underset{d\&Element;D}{\max }\{{\mathrm{\&Sigma;}}_{q\&Element;N_c}{w}_c(p,q)\&times;k\}$

$k=\left\{\begin{array}{cc}1& \mathrm{if}{d}_p\left(q\right)=d\\ 0& \mathrm{otherwise}\end{array}\right.$

S6: generate disparity map and Output rusults.Parallax value in the final parallax matrix is mapped to corresponding gray space [0,255], mapping ratio t=4, the mapping result of parallax value is as follows among the parallax collection D:

0×4＝0 1×4＝4 ... 13×4＝52 14×4＝56

15×4＝60 16×4＝64 ... 28×4＝112 29×4＝116

30×4＝120 31×4＝124 ... 43×4＝172 44×4＝176

45×4＝180 46×4＝184 ... 58×4＝232 59×4＝236

Wherein, parallax value is larger, and more near 255, brighter in disparity map, parallax value is less after the mapping, and is more near 0, darker in disparity map after the mapping.

The present invention is with I _lFor image to be matched, with I _rBe matching image, for each pixel in the left view finds corresponding match point in right view, try to achieve the disparity map of left view.Table 1 is the quantitative comparison of embodiment of the invention result and Yoon self-adaptation support-weight methods and results.As can be seen from Table 1, the present invention is in the unshielding zone, degree of depth discontinuity zone, and All Ranges erroneous matching rate all the erroneous matching rate than Yoon self-adaptation support-weight method is low, and the Stereo matching time of the present invention is about 1/20 of Yoon self-adaptation support-weight method match time, and matching speed is faster.

Table 1

Necessary be pointed out that at this, top embodiment just is used for further setting forth the present invention, so that those of ordinary skill in the art understands the present invention better.The present invention has disclosed its first-selected embodiment by literal; but but can understand wherein optimization and alterability by reading these technology explanatory notes; and improve not departing from scope and spirit of the present invention, but such improvement should still belong to the protection domain of claim of the present invention.

Claims (3)

1. a quick self-adapted support-weight binocular vision solid matching method is characterized in that, may further comprise the steps:

S1, read existing binocular image to be matched to I _lAnd I _r, size and the Color Channel information of acquisition image, wherein, I _lThe expression left view is image to be matched, I _rThe expression right view is matching image;

S2, calculating I _l, I _rCoupling cost between middle pixel comprises:

S21, treat matching image I _l, determine square support window N, calculate and support center pixel and the gray scale difference diff (p that supports pixel in the window, q), wherein, diff (p, q)=and I (p)-I (q), I (p) and I (q) are respectively central pixel point p and support the gray-scale value of pixel q;

S22, according to S21 gained gray scale difference diff (p, q), with each pixel definition is in 5 grades in the square support window N, five grades are as follows:

Wherein, s and t are the threshold values that rule of thumb arranges, and satisfy this principle that affects that reduces as far as possible picture noise;

S23, statistics draw initial similarity measure value S _d:

Make fuz represent the numerical matrix that obtains after the order conversion, matrix size is identical with square support window N size, $\mathrm{fuz}=\left\{\begin{array}{cc}-2& \mathrm{diff}(p,q)<-s\\ -1& -s\&le;\mathrm{diff}(p,q)<-t\\ 0& -t\&le;\mathrm{diff}(p,q)\&le;t\\ 1& t<\mathrm{diff}(p,q)\&le;s\\ 2& s<\mathrm{diff}(p,q)\end{array}\right.,$ Respectively to I _lIn pixel to be matched and I _rMiddle candidate matches pixel carries out the order conversion, can obtain two order transformation matrix fuz _lAnd fuz _r, statistics order transformation matrix fuz _lWith order transformation matrix fuz _rCorrespondence position has the number of same levels in square support window, obtains initial similarity measure value S _d, wherein, $S_d={\mathrm{\&Sigma;}}_{q\&Element;N}m,$ $m=\left\{\begin{array}{cc}1& \mathrm{if}{\mathrm{fuz}}_l={\mathrm{fuz}}_r\\ 0& \mathrm{otherwise}\end{array}\right.,$ M represents order transformation matrix fuz _lAnd fuz _rWhether have same levels at correspondence position, if identical, m=1 then, otherwise m=0;

Initial similarity measure value corresponding to each parallax value d ∈ D in the square statistic window M of n * n in S24, the statistics image to be matched centered by the pixel to be matched obtains the coupling cost between the candidate matches pixel in pixel to be matched and the matching image according to ERT similarity measurement function

Wherein, d represents pixel to be matched and candidate matches pixel parallax in the horizontal direction, D={d _Min... d _Max, $C_{\mathrm{ERT}}(q,{\stackrel{\&OverBar;}{q}}_d)=\underset{n\&times;n}{\mathrm{\&Sigma;}}{S}_d\left(q\right);$

S3, weighting polymerization coupling cost comprise:

S31, calculating support-weight w (p, q): utilize color similarity and how much proximities, calculate the support-weight w (p, q) that the interior support of Matching supporting window pixel q treats matched pixel p, w (p, q)=f _s(Δ c _Pq) f _p(Δ g _Pq),

Wherein, f _s(Δ c _Pq) represent by the definite cluster intensity of color similarity, f _p(Δ g _Pq) represent by how much definite cluster intensity of proximity, Δ c _PqRepresent two pixel color c _pAnd c _qAt the Euclidean distance of RGB color space, c _p=[R _p, G _p, B _p], c _q=[R _q, G _q, B _q], ${\mathrm{\&Delta;c}}_{\mathrm{pq}}=\sqrt{{(R_p-R_q)}^2+{(G_p-G_q)}^2+{(B_p-B_q)}^2},$ Δ g _PqThe Euclidean distance on the locus of expression center pixel and support pixel, establishing pixel p is p (x, y) at the coordinate of image area, pixel q is q (x', y') at the coordinate of image area, then

γ _c, γ _pBe user-specified parameters, be used for respectively the impact on the support-weight size of adjustable colors similarity and how much proximities;

S32, according to S24 gained coupling cost

With S31 gained support-weight w (p, q), weighting polymerization coupling cost obtains: $E(p,{\stackrel{\text{-}}{p}}_d)=\frac{{\mathrm{\&Sigma;}}_{q\&Element;N_P,\stackrel{\&OverBar;}{q}\&Element;N_{{\stackrel{\&OverBar;}{p}}_d}}w(p,q)c_{\mathrm{ERT}}(q,{\stackrel{\&OverBar;}{q}}_d)}{{\mathrm{\&Sigma;}}_{q\&Element;N_P,\stackrel{\&OverBar;}{q}\&Element;N_{{\stackrel{\&OverBar;}{p}}_d}}w(p,q)},$ Wherein,

The matched pixel point that represents respectively image pixel p to be matched, q correspondence in matching image when parallax is d, N _pSupport window size in the expression reference picture,

Corresponding support window size in the expression target image, and

S4, calculating initial parallax: gained weighting polymerization coupling cost among the S3 is adopted suboptimization method WTA(Winner-Take-All, the victor is a king), draw the maximum weighted polymerization result, the parallax value that the maximum weighted polymerization result is corresponding is the initial parallax d of this pixel _p, the initial parallax result of each pixel is saved in the initial parallax matrix, and the initial parallax matrix is: $d_p=\arg \underset{d\&Element;D}{\max }E(p,{\stackrel{\&OverBar;}{p}}_d);$

S5, S4 gained initial parallax is proofreaied and correct, is obtained final parallax matrix, comprising:

S51, definite correcting window N centered by pixel p to be corrected _c, be that each pixel is distributed a suitable support-weight w adaptively in the correction window according to color similarity and how much proximities _c,

The initial parallax of all pixels distributes in S52, the observation correction window, the number of times that statistical parallax d ∈ D occurs in correction window, and the number of times that in correction window, occurs of each parallax value of polymerization d and corresponding weights, the parallax that maximum polymerization result is corresponding then is the final parallax d of pixel to be corrected _{P_final}, and the result is saved in final parallax matrix, wherein $d_{p\_\mathrm{final}}=\arg \underset{d\&Element;D}{\max }\{{\mathrm{\&Sigma;}}_{q\&Element;N_c}{w}_c(p,q)\&times;k\},$ $k=\left\{\begin{array}{cc}1& \mathrm{if}{d}_p\left(q\right)=d\\ 0& \mathrm{otherwise}\end{array}\right.,$ K represents whether the initial parallax of pixel equals to treat statistical parallax d in the correction window, if equate, and k=1 then, otherwise k=0;

S6, generation disparity map, Output rusults: with the final parallax value d of S5 gained _{P_final}Be mapped to corresponding gray space [0,255], the mapping ratio is t, obtains representing the gray level image of parallax information.

2. a kind of quick self-adapted support-weight binocular vision solid matching method according to claim 1 is characterized in that: s＜t described in the S22.

3. a kind of quick self-adapted support-weight binocular vision solid matching method according to claim 1 is characterized in that: pixel should be as far as possible from same depth in the described correction window of S51, and the square correction window of self-adaptation support-weight satisfies this condition.