CN105160679A - Local three-dimensional matching algorithm based on combination of adaptive weighting and image segmentation - Google Patents

Abstract

The invention provides a local three-dimensional matching algorithm based on combination of adaptive weighting and image segmentation. The algorithm comprises: one of a left view and a right view is selected as a reference image; Census conversion is carried out on all pixel points of the left view and the right view so as to obtain matching cists of all pixel points in the reference image under different optical parallaxes d; support windows are established, a reference segmentation unit of each pixel point is established in one support window by using each pixel point of the reference image as a center, a non-reference segmentation unit of each pixel point is established in one support window by using each pixel point of a non-reference image as a center, and a weight value of the central pixel point in each support window is calculated; and according to the weight values, an optimum parallax of the left view and the right view is obtained.

Description

Based on the sectional perspective matching algorithm that adaptive weighting combines with Iamge Segmentation

Technical field

The present invention relates to the Stereo Matching Technology in a kind of binocular stereo vision, particularly a kind of sectional perspective matching algorithm of the adaptive weighting based on improving.

Background technology

Stereo matching as most important in binocular vision be also the most scabrous problem, be always various countries researchist pay close attention to focus, its object is to the optimum parallax value finding left and right view.Real-time and high precision weigh the standard of Stereo Matching Algorithm quality, existing algorithm Problems existing is that the matching algorithm that real-time is good cannot reach excellent effect in degree of accuracy, and the very high algorithm of degree of accuracy sacrifices a large amount of operation time for cost.Stereo Matching Algorithm can be decomposed into 2 committed steps: Matching power flow calculates, and Matching power flow is polymerized.According to the difference of parallax calculation method, Stereo Matching Algorithm can be divided into again sectional perspective matching algorithm and overall Stereo Matching Algorithm.The precision of the ratio of precision sectional perspective coupling of overall situation Stereo matching is high, but is but difficult to the requirement meeting real-time, and the overall Stereo Matching Algorithm used at present relies on high performance hardware device to improve speed.Thus the application of sectional perspective matching algorithm is more extensive.

Summary of the invention

The object of the present invention is to provide a kind of sectional perspective matching algorithm combined with Iamge Segmentation based on adaptive weighting, comprising:

Choose left and right view wherein a width figure as benchmark image;

Respectively Census conversion is carried out to all pixels of left and right view, obtains the Matching power flow of each pixel under different parallax d in benchmark image;

Set up and support window, centered by each pixel of benchmark image, the benchmark cutting unit of each pixel is set up in support window, supporting the non-referenced cutting unit setting up each pixel in window centered by each pixel of non-referenced image, calculate the weighted value that each supports the central pixel point in window;

The optimum parallax of left and right view is obtained according to weighted value.

The present invention compared with prior art, has the following advantages: the present invention is on the basis of Stereo Matching Algorithm, and adopt the adaptive weighting cost algorithms improved to realize the polymerization of Matching power flow, real-time, accuracy is high.

Below in conjunction with Figure of description, the present invention is described further.

Accompanying drawing explanation

Fig. 1 is method flow diagram of the present invention.

Fig. 2 is the method flow diagram that Census of the present invention converts.

Embodiment

Video camera sustained height one the first from left right side of the same model of having demarcated two is parallel to be placed on directly over target scene to be captured, obtain left and right view, choose left and right view wherein a width figure as benchmark image, the present invention is being again that figure is for benchmark image, obtain the optimum parallax of sectional perspective matching algorithm as follows, as shown in Figure 1:

Step S101, carries out Census conversion respectively to all pixels of left and right view, obtains the Matching power flow of each pixel under different parallax d in benchmark image;

Step S102, set up and support window, centered by each pixel of benchmark image, the benchmark cutting unit of each pixel is set up in support window, supporting the non-referenced cutting unit setting up each pixel in window centered by each pixel of non-referenced image, calculate the weighted value that each supports the central pixel point in window;

Step S103, obtains the optimum parallax of left and right view according to weighted value.

Composition graphs 2, the implementation of step S101 is as follows:

Step S1011, choosing right view is benchmark image, centered by a pixel of right figure, sets up Census window, compares the magnitude relationship of the pixel in Census window and the pixel value between central pixel point, and carries out binary conversion treatment by δ function

$\mathrm{\δ}(m_c^r,{\stackrel{\‾}{m}}_a^r)=\left\{\begin{array}{cc}0& m_c^r\≤{\stackrel{\‾}{m}}_a^r\\ 1& m_c^r>{\stackrel{\‾}{m}}_a^r\end{array}\right.$

Wherein, for central pixel point pixel value in Census window in right view, for the pixel value of rest of pixels point in Census window in right view, a is rest of pixels point index value.

Step S1012, sets up slip Census window centered by left view one pixel, compares the magnitude relationship of the pixel value between the pixel in slip Census window and the central pixel point in this Census window,

$\mathrm{\δ}(m_c^l,{\stackrel{\‾}{m}}_a^l)=\left\{\begin{array}{cc}0& m_c^l\≤{\stackrel{\‾}{m}}_a^l\\ 1& m_c^l>{\stackrel{\‾}{m}}_a^l\end{array}\right.$

Wherein, slip Census window size is identical with Census window size in step 11, for central point pixel value in Census window in left view, for the pixel value of rest of pixels point in this Census window in left view, a is rest of pixels point index value.

Step S1013, carries out Census conversion respectively to left and right view, obtains a width Census image.Census conversion is the non-parametric transformations based on mathematical statistics.It is by setting up a window centered by reference image vegetarian refreshments, then the pixel in statistical window and the magnitude relationship between reference image vegetarian refreshments, and what value characterized is numerical relation between reference image vegetarian refreshments and neighborhood territory pixel point.Census conversion is as shown in formula (1):

C(x,y)＝Bitstringδ(I(x,y),I(x+i,y+j))

$i\∈\[-\frac{M}{2},\frac{M}{2}\],j\∈\[-\frac{N}{2},\frac{N}{2}\]$

In formula (1), Bitstring represents the pixel value in window is arranged in bit strings, I (x, y) for coordinate in image is (x, the grey scale pixel value of point y), I (x+i, y+j) represent that decentering point distance is (i, j) grey scale pixel value of place's point, C (x, y) denotation coordination is (x, the Census transformed value of point y), M × N represents the size of the rectangular window centered by (x, y), is called Census window.

Step S1014, the HAMMING distance between Calculation Basis character string and each compare string string

$S(p,q)=HAMMING\left(C\right(p),C\left(q\right))=C\left(p\right)\⊕C\left(q\right)$

Wherein, p is the central point of Census window in right view, q be in left view under d parallax with p Corresponding matching point, C (*) is Census transformed value, and S (p, q) represents the HAMMING distance of p, q 2 under parallax d condition;

Step S1015, changes the value of d, namely scans left view with slip Census window, repeats step 12 ~ 14, until all pixels of left view are completed by scanning;

Step S1016, all pixels in traversal right view, repeat step S1011 ~ S1015.

Step S1017, calculates the Matching power flow of Census image.Matching power flow is used to judge that whether pixel in right figure is the standard of the match point of pixel in left figure, and the general common trait finding pre-matching point in the two width images of left and right judges.

$e{(p,q)}_d=S{(p,q)}_d+\underset{p_c,q_c\∈{\mathrm{window}}_{Census}}{\Σ}S{(p_c,q_c)}_d$

All Matching power flow calculating is carried out to all unique points in right figure, obtains one group of Matching power flow value.Wherein, p _crepresent except excentral point in the Census window centered by p, q _crepresent p under parallax d condition _ccorresponding match point.

The implementation of step S102 is as follows:

Step S1021, with right figure for benchmark image, sets up the support window that size is M' × N' in right figure centered by pixel p, the point supported in window beyond p point is designated as in left figure, centered by pixel q, the M' × N' setting up formed objects supports window, and the point supported in window beyond q point is designated as wherein q be in left view under d parallax with p Corresponding matching point;

Step S1022, adopts following condition to set up benchmark segmentation cell S to benchmark image _p:

$\begin{array}{c}\begin{array}{ccc}{D}_c(p,\stackrel{\&OverBar;}{p})<{\mathrm{\&tau;}}_1& if& 0<D_s(p,\stackrel{\&OverBar;}{p})<l_1\end{array}\\ \begin{array}{ccc}{D}_c(p,\stackrel{\&OverBar;}{p})<{\mathrm{\&tau;}}_2& if& l_1<D_s(p,\stackrel{\&OverBar;}{p})<l_2\end{array}\end{array}$

Support the pixel meeting above-mentioned condition in window same cutting unit is belonged to central pixel point p, wherein, be pixel p and color difference, be pixel p and distance difference, τ ₁, τ ₂color threshold, l ₁, l ₂distance threshold, τ ₁> τ ₂, l ₁<l ₂, τ ₁, τ ₂span be [20,50], l ₁, l ₂span be support [1/4,1/2] of window size, and the computing formula of color difference and distance difference is as follows:

$D_c(p,\stackrel{\&OverBar;}{p})=I_p-I_{\stackrel{\&OverBar;}{p}}$

$D_s(p,\stackrel{\&OverBar;}{p})=\sqrt{{(x_p-x_{\stackrel{\&OverBar;}{p}})}^2+{(y_p-y_{\stackrel{\&OverBar;}{p}})}^2}$

Wherein, (x _p, y _p) be the pixel coordinate of p point, for the pixel coordinate of point;

Step S1023, adopts following condition to set up non-referenced cutting unit S to non-referenced image _q:

$\begin{array}{c}\begin{array}{ccc}{D}_c(q,\stackrel{\&OverBar;}{q})<{\mathrm{\&tau;}}_1& if& 0<D_s(q,\stackrel{\&OverBar;}{q})<l_1\end{array}\\ \begin{array}{ccc}{D}_c(q,\stackrel{\&OverBar;}{q})<{\mathrm{\&tau;}}_2& if& l_1<D_s(q,\stackrel{\&OverBar;}{q})<l_2\end{array}\end{array}$

Support the pixel meeting above-mentioned condition in window same cutting unit is belonged to central pixel point q, wherein, be pixel q and color difference, be pixel q and distance difference, τ ₁, τ ₂color threshold, l ₁, l ₂distance threshold, τ ₁> τ ₂, l ₁<l ₂, τ ₁, τ ₂span be [20,50], l ₁, l ₂span be support [1/4,1/2] of window size, and the computing formula of color difference and distance difference is as follows:

$D_c(q,\stackrel{\&OverBar;}{q})=I_q-I_{\stackrel{\&OverBar;}{q}}$

$D_s(q,\stackrel{\&OverBar;}{q})=\sqrt{{(x_q-x_{\stackrel{\&OverBar;}{q}})}^2+{(y_q-y_{\stackrel{\&OverBar;}{q}})}^2}$

Wherein, (x _q, y _q) be the pixel coordinate of q point, for the pixel coordinate of point.

The method of step S103 is as follows:

Step S1031, calculates the weighted value of the pixel supported in window

$w(q,\stackrel{\&OverBar;}{q})=\left\{\begin{array}{cc}1& q\&Element;S_q\\ \exp (-\frac{D_c\left(q,\stackrel{\&OverBar;}{q}\right)}{{\mathrm{\&gamma;}}_c})& otherwise\end{array}\right.$

$w(p,\stackrel{\&OverBar;}{p})=\left\{\begin{array}{cc}1& p\&Element;S_p\\ \exp (-\frac{D_c\left(p,\stackrel{\&OverBar;}{p}\right)}{{\mathrm{\&gamma;}}_c})& otherwise\end{array}\right.$

Wherein, in benchmark image, set up the support window that size is M' × N' centered by pixel p, the point supported in window beyond p point is designated as in left figure, centered by pixel q, the M' × N' setting up formed objects supports window, and the point supported in window beyond q point is designated as wherein q be in left view under d parallax with p Corresponding matching point, for the weight of pixel in benchmark image, for the weight of pixel in benchmark image, S _pfor benchmark cutting unit, S _qfor non-referenced cutting unit, be pixel p and color difference, be pixel q and color difference, γ _cbe similarity threshold, its span is [10,20];

Step S1032, in order to improve the precision of final Matching power flow, considers that weighted value in left figure and right figure Census window is to Matching power flow e (p, q) simultaneously _doptimized by following formula

$E{(p,q)}_d=\frac{\&Sigma;w(p,\stackrel{\&OverBar;}{p})w(q,\stackrel{\&OverBar;}{q})e{(\stackrel{\&OverBar;}{p},\stackrel{\&OverBar;}{q})}_d}{\&Sigma;w(p,\stackrel{\&OverBar;}{p})w(q,\stackrel{\&OverBar;}{q})}$

Wherein, according to the cost computing formula of the pixel in step 1, in image, support the point outside some p in window matching power flow be

$e{(\stackrel{\&OverBar;}{p},\stackrel{\&OverBar;}{q})}_d=S{(\stackrel{\&OverBar;}{p},\stackrel{\&OverBar;}{q})}_d+\underset{\stackrel{\&OverBar;}{p_c},\stackrel{\&OverBar;}{q_c}\&Element;{\mathrm{window}}_{Census}}{\&Sigma;}S{(\stackrel{\&OverBar;}{p_c},\stackrel{\&OverBar;}{q_c})}_d$

Step S1033, repeats step S1032, under different parallax d condition, to reference view carries out Matching power flow optimization a little;

Step S1034, the Matching power flow value putting the optimization obtained under different parallax according to each in reference view sorts, and the parallax of wherein maximal value representative is the optimum parallax of this point.

Claims (5)

1., based on the sectional perspective matching algorithm that adaptive weighting combines with Iamge Segmentation, it is characterized in that, comprising:

Choose left and right view wherein a width figure as benchmark image;

Respectively Census conversion is carried out to all pixels of left and right view, obtains the Matching power flow of each pixel under different parallax d in benchmark image;

Set up and support window, centered by each pixel of benchmark image, the benchmark cutting unit of each pixel is set up in support window, supporting the non-referenced cutting unit setting up each pixel in window centered by each pixel of non-referenced image, calculate the weighted value that each supports the central pixel point in window;

The optimum parallax of left and right view is obtained according to weighted value.

2. sectional perspective matching algorithm according to claim 1, is characterized in that, the Census change of each pixel described comprises:

Step 11, choosing right view is benchmark image, centered by a pixel of right figure, sets up Census window, compares the magnitude relationship of the pixel in Census window and the pixel value between central pixel point, and carries out binary conversion treatment by δ function

$\mathrm{\&delta;}(m_c^r,{\stackrel{\&OverBar;}{m}}_a^r)=\left\{\begin{array}{cc}0& m_c^r\&le;{\stackrel{\&OverBar;}{m}}_a^r\\ 1& m_c^r>{\stackrel{\&OverBar;}{m}}_a^r\end{array}\right.$

Wherein, for central pixel point pixel value in Census window in right view, for the pixel value of rest of pixels point in Census window in right view, a is rest of pixels point index value;

Step 12, sets up slip Census window centered by left view one pixel, compares the magnitude relationship of the pixel value between the pixel in slip Census window and the central pixel point in this Census window, and carries out binary conversion treatment by δ function

$\mathrm{\&delta;}(m_c^l,{\stackrel{\&OverBar;}{m}}_a^l)=\left\{\begin{array}{cc}0& m_c^l\&le;{\stackrel{\&OverBar;}{m}}_a^l\\ 1& m_c^l>{\stackrel{\&OverBar;}{m}}_a^l\end{array}\right.$

Wherein, slip Census window size is identical with Census window size in step 11, for central point pixel value in Census window in left view, for the pixel value of rest of pixels point in this Census window in left view, a is rest of pixels point index value;

Step 13, the order from left to right first up and then down according to the following formula to the pixel value in each Census window in the view of left and right after δ functional transformation is arranged in bit strings, right view be benchmark character string, left view be compare string string

$C(x,y)=\underset{i\&Element;\&lsqb;-\frac{M}{2},\frac{M}{2}\&rsqb;,j\&Element;\&lsqb;-\frac{N}{2},\frac{N}{2}\&rsqb;}{Bitstring}\mathrm{\&delta;}(I(x,y),I(x+i,y+j))$

Wherein, I (x, y) is the grey scale pixel value of the point of (x, y) for coordinate in image, I (x+i, y+j) represent that decentering point distance is the grey scale pixel value of (i, j) place point, C (x, y) denotation coordination is (x, the Census transformed value of point y), M × N represents the size of the Census window centered by (x, y);

Step 14, the HAMMING distance between Calculation Basis character string and each compare string string

$S(p,q)=HAMMING\left(C\right(p),C\left(q\right))=C\left(p\right)\&CirclePlus;C\left(q\right)$

Wherein, p is the central point of Census window in right view, q be in left view under d parallax with p Corresponding matching point, C (*) is Census transformed value, and S (p, q) represents the HAMMING distance of p, q 2 under parallax d condition;

Step 15, changes the value of d, namely scans left view with slip Census window, repeats step 12 ~ 14, until all pixels of left view are completed by scanning;

Step 16, all pixels in traversal right view, repeat step 11 ~ 15.

3. sectional perspective matching algorithm according to claim 2, is characterized in that, adopts following formula to obtain the Matching power flow of each pixel under different parallax d in benchmark image.

$e{(p,q)}_d=S{(p,q)}_d+\underset{p_c,q_c\&Element;{\mathrm{window}}_{Census}}{\&Sigma;}S{(p_c,q_c)}_d$

Wherein, p _crepresent except excentral point in the Census window centered by p, q _crepresent p under parallax d condition _ccorresponding match point.

4. sectional perspective matching algorithm according to claim 1, is characterized in that, describedly centered by each pixel of benchmark image, sets up cutting unit comprise:

Step 21, with right figure for benchmark image, sets up the support window that size is M' × N' in right figure centered by pixel p, the point supported in window beyond p point is designated as in left figure, centered by pixel q, the M' × N' setting up formed objects supports window, and the point supported in window beyond q point is designated as wherein q be in left view under d parallax with p Corresponding matching point;

Step 22, adopts following condition to set up benchmark segmentation cell S to benchmark image _p:

$\begin{array}{ccc}{D}_c(p,\stackrel{\&OverBar;}{p})<{\mathrm{\&tau;}}_1& if& 0<D_s(p,\stackrel{\&OverBar;}{p})<l_1\end{array}$

$\begin{array}{ccc}{D}_c(p,\stackrel{\&OverBar;}{p})<{\mathrm{\&tau;}}_2& if& l_1<D_s(p,\stackrel{\&OverBar;}{p})<l_2\end{array}$

Support the pixel meeting above-mentioned condition in window same cutting unit is belonged to central pixel point p, wherein, be pixel p and color difference, be pixel p and distance difference, τ ₁, τ ₂color threshold, l ₁, l ₂distance threshold, τ ₁> τ ₂, l ₁<l ₂, τ ₁, τ ₂span be [20,50], l ₁, l ₂span be support [1/4,1/2] of window size, and the computing formula of color difference and distance difference is as follows:

$D_c(p,\stackrel{\&OverBar;}{p})=I_p-I_{\stackrel{\&OverBar;}{p}}$

$D_s(p,\stackrel{\&OverBar;}{p})=\sqrt{{(x_p-x_{\stackrel{\&OverBar;}{p}})}^2+{(y_p-y_{\stackrel{\&OverBar;}{p}})}^2}$

Wherein, (x _p, y _p) be the pixel coordinate of p point, for the pixel coordinate of point;

Step 23, adopts following condition to set up non-referenced cutting unit S to non-referenced image _q:

$\begin{array}{ccc}{D}_c(q,\stackrel{\&OverBar;}{q})<{\mathrm{\&tau;}}_1& if& 0<D_s(q,\stackrel{\&OverBar;}{q})<l_1\end{array}$

$\begin{array}{ccc}{D}_c(q,\stackrel{\&OverBar;}{q})<{\mathrm{\&tau;}}_2& if& l_1<D_s(q,\stackrel{\&OverBar;}{q})<l_2\end{array}$

Support the pixel meeting above-mentioned condition in window same cutting unit is belonged to central pixel point q, wherein, be pixel q and color difference, be pixel q and distance difference, τ ₁, τ ₂color threshold, l ₁, l ₂distance threshold, τ ₁> τ ₂, l ₁<l ₂, τ ₁, τ ₂span be [20,50], l ₁, l ₂span be support [1/4,1/2] of window size, and the computing formula of color difference and distance difference is as follows:

$D_c(q,\stackrel{\&OverBar;}{q})=I_q-I_{\stackrel{\&OverBar;}{q}}$

$D_s(q,\stackrel{\&OverBar;}{q})=\sqrt{{(x_q-x_{\stackrel{\&OverBar;}{q}})}^2+{(y_q-y_{\stackrel{\&OverBar;}{q}})}^2}$

Wherein, (x _q, y _q) be the pixel coordinate of q point, for the pixel coordinate of point.

5. sectional perspective matching algorithm according to claim 1, is characterized in that, the finding method of optimum parallax comprises:

Step 31, calculates the weighted value of the pixel supported in window

$w(q,\stackrel{\&OverBar;}{q})=\left\{\begin{array}{cc}1& q\&Element;S_q\\ \exp (-\frac{D_c\left(q,\stackrel{\&OverBar;}{q}\right)}{{\mathrm{\&gamma;}}_c})& otherwise\end{array}\right.$

$w(p,\stackrel{\&OverBar;}{p})=\left\{\begin{array}{cc}1& p\&Element;S_p\\ \exp (-\frac{D_c\left(p,\stackrel{\&OverBar;}{p}\right)}{{\mathrm{\&gamma;}}_c})& otherwise\end{array}\right.$

Wherein, in benchmark image, set up the support window that size is M' × N' centered by pixel p, the point supported in window beyond p point is designated as in left figure, centered by pixel q, the M' × N' setting up formed objects supports window, and the point supported in window beyond q point is designated as wherein q be in left view under d parallax with p Corresponding matching point, for the weight of pixel in benchmark image, for the weight of pixel in benchmark image, S _pfor benchmark cutting unit, S _qfor non-referenced cutting unit, be pixel p and color difference, be pixel q and color difference, γ _cbe similarity threshold, its span is [10,20];

Step 32, to Matching power flow e (p, q) _doptimized by following formula

$E{(p,q)}_d=\frac{\&Sigma;w(p,\stackrel{\&OverBar;}{p})w(q,\stackrel{\&OverBar;}{q})e{(\stackrel{\&OverBar;}{p},\stackrel{\&OverBar;}{q})}_d}{\&Sigma;w(p,\stackrel{\&OverBar;}{p})w(q,\stackrel{\&OverBar;}{q})}$

Wherein, according to the cost computing formula of the pixel in step 1, in image, support the point outside some p in window matching power flow be

$e{(\stackrel{\&OverBar;}{p},\stackrel{\&OverBar;}{q})}_d=S{(\stackrel{\&OverBar;}{p},\stackrel{\&OverBar;}{q})}_d+\underset{\stackrel{\&OverBar;}{p_c},\stackrel{\&OverBar;}{q_c}\&Element;{\mathrm{window}}_{Census}}{\&Sigma;}S{(\stackrel{\&OverBar;}{p_c},\stackrel{\&OverBar;}{q_c})}_d$

Step 33, repeats step 32, under different parallax d condition, to reference view carries out Matching power flow optimization a little;

Step 34, the Matching power flow value putting the optimization obtained under different parallax according to each in reference view sorts, and the parallax of wherein maximal value representative is the optimum parallax of this point.