CN104820991A - Multi-soft-constraint stereo matching method based on cost matrix - Google Patents

Abstract

The invention discloses a multi-soft-constraint stereo matching method based on a cost matrix. The method comprises the following steps: calculating a three-dimensional matching cost matrix at first, then carrying out multiple-dimension downsampling, and forming a cost matrix pyramid; carrying out the corresponding multiple-dimension downsampling of an image, forming an image pyramid, and carrying out segmentation of all image layers; carrying out cost accumulation of self-adaption weight layer by layer and cost accumulation under the voting-type segmentation constraint secondly, and achieving the cost accumulation under the multiple-dimension constraint through transmitting the cost accumulation results of the upper layer to the lower cost matrix; and achieving stereo matching through combining a reliable point cost diffusion method in a bottom layer cost matrix. The method improves the stability and reliability of matching results, solves the matching problems of weak texture and repeated texture regions and parallax error discontinuous places, and can be used for improving modeling, related to the engineering application of stereo matching, based on images.

Description

A Multiple Soft Constrained Stereo Matching Method Based on Cost Matrix

technical field

The invention belongs to the technical field of computational vision and photogrammetry, and relates to a stereo matching method based on a cost matrix, in particular to a stereo matching method based on a cost matrix with multiple soft constraints.

Background technique

Stereo matching is a fundamental and key problem in the fields of computational vision and photogrammetry. The concept of stereo matching was first proposed in the field of photogrammetry to solve the problem of automatic mapping of digital aerial photogrammetry. Stereo matching is also a key issue in the field of computer vision, affecting modeling of the human visual system, robot navigation and manipulation, 3D model building, and mixing real-world actions with computer-generated images.

Stereo matching is a pathological problem, especially for the lack of texture and repeated texture regions, and the mismatching problems of discontinuous disparity have been difficult to solve comprehensively. In order to achieve better matching results, more prior constraints need to be incorporated reasonably. The matching algorithm generally uses local window-based matching. During the matching process, the local information of the image is used to improve the window matching. At the same time, the pyramid layered matching strategy is used to improve the stability and accuracy. However, these methods often require large matching windows, making it difficult to preserve image details. There are also some algorithms that incorporate image segmentation constraints, but have a strong dependence on image segmentation results. In short, the traditional stereo matching method often directly uses various constraints to reduce the disparity search range, so it is highly dependent on the prior conditions and is prone to mis-matching.

In recent years, there have been some algorithms that achieve matching by improving the cost accumulation method, such as using the adaptive cost accumulation method to improve the local window matching algorithm, using the multi-scale cost accumulation method to solve the matching problem of repeated textures, and using the semi-global cost accumulation method to solve the parallax discontinuity matching problem. The common point of these algorithms is to implement their respective constraints in the cost matrix, but because the constraints included are not sufficient, it is difficult to solve the mismatching problem comprehensively.

Contents of the invention

The present invention mainly solves the problem of excessive dependence on the priori condition constraints existing in the prior art; it provides a multiple soft constraint cost accumulation method based on the cost matrix that can simultaneously incorporate multiple priori constraints to complete stereo matching, It can effectively solve the problem of lack of texture and repeated texture areas and the mismatching of edges.

The technical scheme adopted in the present invention is: a kind of multiple soft constraint stereo matching method based on cost matrix, it is characterized in that, comprises the following steps:

Step 1: Select one of the epipolar images in the original image set as the reference image, and use AD-Census-Sobel as the similarity measure for each pixel, calculate the matching cost of each candidate disparity value, and generate a three-dimensional matching cost matrix;

Step 2: Perform multi-scale downsampling on the cost matrix obtained in step 1 to form a cost matrix pyramid;

Step 3: Perform corresponding multi-scale downsampling on the reference image to form an image pyramid;

Step 4; Carry out image segmentation respectively to the images of each layer of the image pyramid obtained in step 3;

Step 5: For the cost matrix pyramid obtained in step 2, start from the top-level cost matrix to accumulate the cost of adaptive weights for each layer of cost matrix;

Step 6: According to the image segmentation results described in step 4, the cost matrix processed in step 5 is further subjected to cost accumulation under the "voting" segmentation constraint;

Step 7: Transfer the cost accumulation result of this layer to the lower layer cost matrix;

Step 8: Repeat steps 5 to 7 on the pyramid cost matrix from top to bottom until the original cost matrix, and finally perform steps 5 and 6 on the original cost matrix;

Step 9: For the cost matrix obtained after processing in step 8, replace the disparity with the minimum cost and matching confidence greater than the predetermined threshold for each pixel as the final disparity, and then use the obtained disparity point as a control in the cost matrix for unmatched Points for cost diffusion, and finally use the "winner takes all" method to generate a disparity map;

Step 10: Select another image in the original image set as the reference image, repeat the above steps 1 to 9 to generate another disparity map, and detect the mismatching points by comparing the disparity consistency of the corresponding pixels of the two disparity maps to obtain Remove the disparity map of mismatching points; perform parallax interpolation on the disparity map of removed mismatching points, fill in the black holes caused by mismatching, and generate a complete disparity map; use the complete disparity map and the outer orientation elements of the image to calculate the depth map and digital according to the front intersection principle surface model.

As a preference, in step 1,

The definition of the AD-Census-Sobel similarity measure is as follows:

cost(p,d)＝exp(α*AD(p,d)+β*Census(p,d)+γ*Sobel(p,d)) (Formula 1);

Among them, p represents the pixel coordinate p(x, y) on the reference image, d is the parallax value, α, β and γ are weight coefficients, generally required to be positive numbers and α+β+γ=1, AD(p ,d) refers to the absolute value of the gray level difference between the reference image pixel p and its conjugate pixel p+d on the matching image, and Census(p,d) refers to the reference image pixel p and its conjugate pixel on the matching image The Census matching measure value of pixel p+d, Sobel(p,d) refers to the Sobel matching measure value of reference image pixel p and its conjugate pixel p+d on the matching image;

According to formula 1, the matching cost values corresponding to different disparities d are calculated pixel by pixel for a pair of reference images, and a three-dimensional matching cost matrix represented by image abscissa W, ordinate H and disparity value D is formed.

Preferably, the generation method of the cost matrix pyramid described in step 2 is to keep the disparity value D direction unchanged for the three-dimensional matching cost matrix obtained in step 1, and use a Gaussian pyramid in the direction of abscissa W and ordinate H to reduce Sampling, according to this method, a multi-scale cost matrix can be generated after multiple downsampling.

Preferably, the multi-scale down-sampling method described in step 3 is to use a Gaussian pyramid to down-sample the original image in the directions of the abscissa W and the ordinate H, and the image pyramid can be generated through multiple down-sampling according to this method.

Preferably, the image segmentation method described in step 4 is to first adopt the existing image segmentation method, then perform segmentation block clustering after the image segmentation is completed, and perform segmentation again for categories whose number of elements is greater than a predetermined threshold, until each The number of elements of a category is not greater than a predetermined threshold.

As a preference, the cost accumulation method of the adaptive weight described in step 5 is based on the Euclidean distance dist(p,q) and gray value difference between pixel q in a certain window centered on each pixel point p and the point |I _p -I _q |Determine the contribution value of pixel q to the cost of pixel p, so that the contribution of all pixels in the window to the cost of pixel p is accumulated to obtain the final cost value cost(p,d) of p. The specific calculation method is as follows :

$\cos t(p,d)=\underset{q\∈W_p}{\mathrm{\Σ}}\cos t(q,d)*\mathrm{weight}(q,d)$ (Formula II);

weight(q,d)=exp(α*dis(p,q)+β*|I _p -I _q |)

Among them, weight(q,d) represents the weight value of the neighborhood pixels when the cost is accumulated, α and β are the parameters involved in calculating the weight value, corresponding to the Euclidean distance dist(p,q) and the gray value difference| The weight coefficients of I _p -I _q | are generally required to be positive numbers and α+β=1.

As preferably, the specific realization of step 6 includes the following sub-steps:

Step 6.1: For each image segmentation block, add up the total cost corresponding to each disparity value of all pixels in the image segmentation block, and obtain the disparity-cost histogram of the matching cost changing with the disparity value in the entire image segmentation block picture;

Step 6.2: Calculate the mean value of the matching cost, the difference between each disparity value and the mean value and the mean value of these differences according to the histogram, and assign a penalty to the disparity value whose difference is negative and whose absolute value is greater than the mean value of the difference, That is, a negative cost value is assigned, and a disparity-cost penalty histogram corresponding to the disparity-cost histogram described in step 6.1 is obtained in this way;

Step 6.3: Apply the above cost penalty to the image segmentation block pixel by pixel, that is, add the corresponding cost penalty to the disparity value of each pixel according to the disparity-cost penalty histogram described in step 6.2;

The cost penalty is calculated as follows:

$p\left(d\right)=\left\{\begin{array}{cc}0,& c_d>\stackrel{\&OverBar;}{c_d}\\ \mathrm{\&alpha;}*(c_d-\stackrel{\&OverBar;}{c_d})/\stackrel{\&OverBar;}{|c_d-\stackrel{\&OverBar;}{c_d}|},& c_d<\stackrel{\&OverBar;}{c_d}\end{array}\right.$ (Formula 3);

Among them, c _d is the cost of the segmentation block corresponding to the disparity d, and α is the weight coefficient.

As preferably, the layer cost accumulation result described in step 7 is passed to the lower layer cost matrix, and its specific implementation includes the following sub-steps:

Step 7.1: Perform one-dimensional minimum square convolution on the cost value of each pixel within the parallax range; note that the purpose of the convolution here is to make the upper layer matching cost smoother with the parallax change, so the minimum sum convolution can also be used or other convolution methods;

Step 7.2: According to the pyramid pixel correspondence, add the cost value of each pixel in this layer to the cost value of the corresponding disparity of the corresponding pixel in the lower layer.

As a preference, the matching confidence degree described in step 9 is the ratio of the second minimum value of the matching cost to the minimum value, and only the pixels whose matching confidence degree is greater than a predetermined threshold can obtain parallax for the first time; The method of cost diffusion for unmatched points is as follows: mark the matched points and unmatched points, and then subtract a certain penalty value from the unmatched pixels in a certain window neighborhood of each matched point corresponding to the disparity cost of the matched point , as shown in the following formula:

$\cos t(q,d)=\left\{\begin{array}{c}\cos t(q,d)-P,\mathrm{D.}\left(p\right)=d\\ \cos t(q,d),\mathrm{D.}\left(p\right)\&NotEqual;d\end{array}\right.,q\&Element;N_p,\mathrm{D.}\left(q\right)=\mathrm{NULL}$ (style four);

Among them, p represents the matched point, q represents the unmatched point, P represents the penalty value, and D(p) represents the value of the initial disparity map D at point p.

The present invention has the following advantages: local texture structure and gray information are considered in the process of stereo matching, and the distinguishability of points with the same name is enhanced; multi-scale constraints are incorporated to enhance the matching ability of repeated texture regions; segmentation constraints are incorporated to enhance The matching ability of the weak texture area is improved; the cost diffusion is carried out according to the primary reliable matching result, which enhances the matching stability; all the above constraints are completed in the cost matrix, and each constraint is a soft constraint that cannot directly determine the final matching result, which further enhances the Stability and reliability of matching results.

Description of drawings

Fig. 1: is the overall flowchart of the embodiment of the present invention;

Figure 2: a schematic diagram of a multi-scale downsampling method for a cost matrix according to an embodiment of the present invention;

Fig. 3 is a schematic diagram of a cost accumulation method based on a cost matrix "voting" partition constraint according to an embodiment of the present invention.

Detailed ways

In order to facilitate those of ordinary skill in the art to understand and implement the present invention, the present invention will be described in further detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described here are only used to illustrate and explain the present invention, and are not intended to limit this invention.

Please see Fig. 1, a kind of multiple soft constraint stereo matching method based on cost matrix provided by the present invention, comprises the following steps:

Step 1: Select one of the epipolar images in the original image set as the reference image, and use AD-Census-Sobel as the similarity measure for each pixel, calculate the matching cost of each candidate disparity value, and generate a three-dimensional matching cost matrix;

The AD-Census-Sobel similarity measure is defined as follows:

cost(p,d)＝exp(α*AD(p,d)+β*Census(p,d)+γ*Sobel(p,d)) (Formula 1);

Among them, p represents the pixel coordinate p(x, y) on the reference image, d is the parallax value, α, β and γ are weight coefficients, generally required to be positive numbers and α+β+γ=1, AD(p ,d) refers to the absolute value of the gray level difference between the reference image pixel p and its conjugate pixel p+d on the matching image, and Census(p,d) refers to the reference image pixel p and its conjugate pixel on the matching image The Census matching measure value of pixel p+d, Sobel(p,d) refers to the Sobel matching measure value of reference image pixel p and its conjugate pixel p+d on the matching image;

Calculate the matching cost values corresponding to different disparities d pixel by pixel according to the formula 1 pair of reference images, and form a three-dimensional matching cost matrix represented by the image abscissa W, ordinate H and disparity value D, with a size of W*H*D.

Step 2: Perform multi-scale downsampling on the cost matrix obtained in step 1 to form a cost matrix pyramid;

Please see Figure 2. The method of generating the cost matrix pyramid is to keep the disparity value D direction unchanged for the three-dimensional matching cost matrix obtained in step 1, and use the Gaussian pyramid for downsampling in the abscissa W and ordinate H directions. According to this The method can generate a multi-scale cost matrix through multiple downsampling.

Step 3: Carry out corresponding multi-scale down-sampling on the reference image to form an image pyramid; the method of multi-scale down-sampling is to use Gaussian pyramid to down-sample the original image in the direction of abscissa W and ordinate H, according to this method The image pyramid can be generated by multiple downsampling.

Step 4; Carry out image segmentation to the images of each layer of the image pyramid obtained in step 3; the image segmentation method is to first adopt the existing image segmentation method, and then perform segmentation block clustering after the image segmentation is completed, and for its element number greater than The categories of the predetermined threshold are divided again until the number of elements of each category is not greater than the predetermined threshold.

Step 5: For the cost matrix pyramid obtained in step 2, start from the top-level cost matrix to accumulate the cost of adaptive weights for each layer of the cost matrix; the method of cost accumulation for adaptive weights is to use each pixel point p as the center for a certain window The Euclidean distance dist(p,q) and the gray value difference between the pixel q in the pixel q and the point |I _p -I _q | determine the contribution value of the pixel point q to the pixel point p cost, so that all pixels in the window are accumulated for the pixel p The final cost value cost(p,d) of p is obtained by the contribution of the cost. The specific calculation method is as follows:

$\cos t(p,d)=\underset{q\&Element;W_p}{\mathrm{\&Sigma;}}\cos t(q,d)*\mathrm{weight}(q,d)$ (Formula II);

weight(q,d)=exp(α*dis(p,q)+β*|I _p -I _q |)

Among them, weight(q,d) represents the weight value of the neighborhood pixels when the cost is accumulated, α and β are the parameters involved in calculating the weight value, corresponding to the Euclidean distance dist(p,q) and the gray value difference| The weight coefficients of I _p -I _q | are generally required to be positive numbers and α+β=1.

Step 6: According to the image segmentation results described in step 4, the cost matrix processed in step 5 is further subjected to cost accumulation under the "voting" segmentation constraint;

See Figure 3, the specific implementation includes the following sub-steps:

Step 6.1: For each image segmentation block, add up the total cost corresponding to each disparity value of all pixels in the image segmentation block, and obtain the disparity-cost histogram of the matching cost changing with the disparity value in the entire image segmentation block picture;

Step 6.2: Calculate the mean value of the matching cost, the difference between each disparity value and the mean value and the mean value of these differences according to the histogram, and assign a penalty to the disparity value whose difference is negative and whose absolute value is greater than the mean value of the difference, That is, a negative cost value is assigned, and a disparity-cost penalty histogram corresponding to the disparity-cost histogram described in step 6.1 is obtained in this way;

Step 6.3: Apply the above cost penalty to the image segmentation block pixel by pixel, that is, add the corresponding cost penalty to the disparity value of each pixel according to the disparity-cost penalty histogram described in step 6.2;

The cost penalty is calculated as follows:

$p\left(d\right)=\left\{\begin{array}{cc}0,& c_d>\stackrel{\&OverBar;}{c_d}\\ \mathrm{\&alpha;}*(c_d-\stackrel{\&OverBar;}{c_d})/\stackrel{\&OverBar;}{|c_d-\stackrel{\&OverBar;}{c_d}|},& c_d<\stackrel{\&OverBar;}{c_d}\end{array}\right.$ (Formula 3);

Among them, c _d is the cost of the segmentation block corresponding to the disparity d, and α is the weight coefficient.

Step 7: Transfer the cost accumulation result of this layer to the lower layer cost matrix; its specific implementation includes the following sub-steps:

Step 7.1: Perform one-dimensional least square convolution on the cost value of each pixel within the parallax range according to formula 4;

${\cos t}^{\&prime;}\left(d\right)=\underset{d}{\min }(\cos t\left(d\right)+{(d-d^{\&prime;})}^2)$ (style four);

Among them, cost'(d) represents the cost value corresponding to the parallax d after convolution, and d' is other parallax values within the parallax search range;

Step 7.2: According to the pyramid pixel correspondence, add the cost value of each pixel in this layer to the cost value of the corresponding disparity of the corresponding pixel in the lower layer.

Step 8: Repeat steps 5 to 7 on the pyramid cost matrix from top to bottom until the original cost matrix, and finally perform steps 5 and 6 on the original cost matrix;

Step 9: For the cost matrix obtained after processing in step 8, replace the disparity with the smallest cost and the matching confidence _greater than the predetermined threshold for each pixel according to Equation 5 as the final disparity. The ratio of the values, for the first time disparity can be obtained only for pixels whose matching confidence is greater than a predetermined threshold.

(style five);

Then mark the matched point and the unmatched point, and then subtract a certain penalty value P from the disparity cost of the unmatched pixel in the 8 neighborhood of each matched point corresponding to the matched point according to the formula, so that the matched result is converted to the cost The form of the penalty is diffused to the neighborhood of unmatched points.

$\cos t(q,d)=\left\{\begin{array}{c}\cos t(q,d)-P,\mathrm{D.}\left(p\right)=d\\ \cos t(q,d),\mathrm{D.}\left(p\right)\&NotEqual;d\end{array}\right.,q\&Element;N_p,\mathrm{D.}\left(q\right)=\mathrm{NULL}$ (style land);

Among them, p represents the matched point, q represents the unmatched point, P represents the penalty value, and D(p) represents the value of the initial disparity map D at point p.

Finally, the "winner takes all" method shown in formula 7 is used to generate the disparity map.

$\mathrm{D.}\left(p\right)=\arg \underset{d}{\min }\left(\cos t(p,d)\right)$ (type seven);

Step 10: Select another image in the original image set as the reference image, repeat the above steps 1 to 9 to generate another disparity map, and detect the mismatching points by comparing the disparity consistency of the corresponding pixels of the two disparity maps to obtain Remove the disparity map of mismatching points; perform parallax interpolation on the disparity map of removed mismatching points, fill in the black holes caused by mismatching, and generate a complete disparity map; use the complete disparity map and the outer orientation elements of the image to calculate the depth map and digital according to the front intersection principle surface model.

The mathematical measure of left-right disparity consistency detection is:

LRC (p)=|D _LR (p)-D _RL (p+D _LR (p))| (formula eight);

Among them, D _LR represents the matching result with the left image as the reference image, D _RL is the matching result with the right image as the reference image, and the matching points with LRC(p) greater than a certain threshold are considered as illegal points.

Perform occlusion detection and parallax interpolation on the above disparity map, use the disparity of the existing correct matching point to fill the black hole caused by the mis-match of its neighborhood, and generate a complete disparity map;

Among them, the occlusion detection method is: for an illegal point p on the reference image, search along the horizontal epipolar line on the matching image, if there is a parallax d such that D(p+d)-d<T _d , then the point It is a mismatch, otherwise it is an occlusion, as shown in formula 9:

$\mathrm{flag}\left(p\right)=\left\{\begin{array}{c}\mathrm{occluded},\mathrm{D.}(p+d)-d<T_d\\ \mathrm{mismatch},\mathrm{D.}(p+d)-d<T_d\end{array}\right.$ (form nine);

The parallax interpolation method is:

$\mathrm{D.}\left(p\right)=\left\{\begin{array}{c}{\mathrm{D.}}_{\mathrm{low}}(q\&Element;N_p),\mathrm{flag}\left(p\right)=\mathrm{occluded}\\ {\mathrm{D.}}_{\mathrm{median}}(q\&Element;N_p),\mathrm{flag}\left(p\right)=\mathrm{mismatched}\\ \mathrm{D.}\left(p\right),\mathrm{otherwise}\end{array}\right.$ (style pick up);

Among them, D _low (q∈N _p ) means to select a smaller disparity from the neighborhood, and D _median (q∈N _p ) means to choose the median value of the disparity from the neighborhood.

A depth map or a digital surface model can be calculated by forward intersection using the above disparity map and image exterior orientation elements.

It should be understood that the parts not described in detail in this specification belong to the prior art.

It should be understood that the above-mentioned descriptions for the preferred embodiments are relatively detailed, and should not therefore be considered as limiting the scope of the patent protection of the present invention. Within the scope of protection, replacements or modifications can also be made, all of which fall within the protection scope of the present invention, and the scope of protection of the present invention should be based on the appended claims.

Claims (9)

1. a multiple soft constraint stereo matching method based on cost matrix, is characterized in that, comprises the following steps: Step 1: Select one of the epipolar images in the original image set as the reference image, and use AD-Census-Sobel as the similarity measure for each pixel, calculate the matching cost of each candidate disparity value, and generate a three-dimensional matching cost matrix; Step 2: Perform multi-scale downsampling on the cost matrix obtained in step 1 to form a cost matrix pyramid; Step 3: Perform corresponding multi-scale downsampling on the reference image to form an image pyramid; Step 4; Carry out image segmentation respectively to the images of each layer of the image pyramid obtained in step 3; Step 5: For the cost matrix pyramid obtained in step 2, start from the top-level cost matrix to accumulate the cost of adaptive weights for each layer of cost matrix; Step 6: According to the image segmentation results described in step 4, the cost matrix processed in step 5 is further subjected to cost accumulation under the "voting" segmentation constraint; Step 7: Transfer the cost accumulation result of this layer to the lower layer cost matrix; Step 8: Repeat steps 5 to 7 on the pyramid cost matrix from top to bottom until the original cost matrix, and finally perform steps 5 and 6 on the original cost matrix; Step 9: For the cost matrix obtained after processing in step 8, replace the disparity with the minimum cost and matching confidence greater than the predetermined threshold for each pixel as the final disparity, and then use the obtained disparity point as a control in the cost matrix for unmatched Points for cost diffusion, and finally use the "winner takes all" method to generate a disparity map; Step 10: Select another image in the original image set as the reference image, repeat the above steps 1 to 9 to generate another disparity map, and detect the mismatching points by comparing the disparity consistency of the corresponding pixels of the two disparity maps to obtain Remove the disparity map of mismatching points; perform parallax interpolation on the disparity map of removed mismatching points, fill in the black holes caused by mismatching, and generate a complete disparity map; use the complete disparity map and the outer orientation elements of the image to calculate the depth map and digital according to the front intersection principle surface model. 2. the multiple soft constraint stereo matching method based on cost matrix according to claim 1, is characterized in that: in step 1, The definition of the AD-Census-Sobel similarity measure is as follows: cost(p,d)＝exp(α*AD(p,d)+β*Census(p,d)+γ*Sobel(p,d)) (Formula 1); Among them, p represents the pixel coordinate p(x, y) on the reference image, d is the parallax value, α, β and γ are weight coefficients, generally required to be positive numbers and α+β+γ=1, AD(p ,d) refers to the absolute value of the gray level difference between the reference image pixel p and its conjugate pixel p+d on the matching image, and Census(p,d) refers to the reference image pixel p and its conjugate pixel on the matching image The Census matching measure value of pixel p+d, Sobel(p,d) refers to the Sobel matching measure value of reference image pixel p and its conjugate pixel p+d on the matching image; According to formula 1, the matching cost values corresponding to different disparities d are calculated pixel by pixel for a pair of reference images, and a three-dimensional matching cost matrix represented by image abscissa W, ordinate H and disparity value D is formed. 3. the multiple soft constraint stereo matching method based on cost matrix according to claim 2, is characterized in that: the generation method of the cost matrix pyramid described in the step 2 is, the three-dimensional matching cost matrix obtained in the step 1 maintains visual The direction of the difference D remains unchanged, and the Gaussian pyramid is used for downsampling in the direction of the abscissa W and the ordinate H. According to this method, a multi-scale cost matrix can be generated after multiple downsampling. 4. The multiple soft-constrained stereo matching method based on cost matrix according to claim 2, characterized in that: the multi-scale down-sampling method described in step 3 is, for the original image in the abscissa W and ordinate H directions The Gaussian pyramid is used for downsampling, and the image pyramid can be generated by multiple downsampling according to this method. 5. the multiple soft constraint stereo matching method based on cost matrix according to claim 1, is characterized in that: the image segmentation method described in step 4 is to adopt existing image segmentation method at first, then carry out after image segmentation is finished The block clustering is divided, and the category whose number of elements is greater than the predetermined threshold is divided again until the number of elements of each category is not greater than the predetermined threshold. 6. The multiple soft-constrained stereo matching method based on cost matrix according to claim 1, characterized in that: the cost accumulation method of the adaptive weight described in step 5 is, according to each pixel point p as the center within a certain window The Euclidean distance dist(p,q) and the gray value difference between the pixel q of the pixel q and the point |I _p -I _q | determine the contribution value of the pixel point q to the pixel point p cost, so that all pixels of the window are accumulated for the pixel p cost The contribution of p to get the final cost value cost(p,d), the specific calculation method is as follows: $\cos t(p,d)=\underset{q\&Element;W_p}{\mathrm{\&Sigma;}}\cos t(q,d)*\mathrm{weight}(q,d)$ (Formula II); weight(q,d)=exp(α*dis(p,q)+β*|I _p -I _q |) Among them, weight(q,d) represents the weight value of the neighborhood pixels when the cost is accumulated, α and β are the parameters involved in calculating the weight value, corresponding to the Euclidean distance dist(p,q) and the gray value difference| The weight coefficients of I _p -I _q | are generally required to be positive numbers and α+β=1. 7. the multiple soft constraint stereo matching method based on cost matrix according to claim 1, is characterized in that: the concrete realization of step 6 comprises the following sub-steps: Step 6.1: For each image segmentation block, add up the total cost corresponding to each disparity value of all pixels in the image segmentation block, and obtain the disparity-cost histogram of the matching cost changing with the disparity value in the entire image segmentation block picture; Step 6.2: Calculate the mean value of the matching cost, the difference between each disparity value and the mean value and the mean value of these differences according to the histogram, and assign a penalty to the disparity value whose difference is negative and whose absolute value is greater than the mean value of the difference, That is, a negative cost value is assigned, and a disparity-cost penalty histogram corresponding to the disparity-cost histogram described in step 6.1 is obtained in this way; Step 6.3: Apply the above cost penalty to the image segmentation block pixel by pixel, that is, add the corresponding cost penalty to the disparity value of each pixel according to the disparity-cost penalty histogram described in step 6.2; The cost penalty is calculated as follows: $P\left(d\right)=\left\{\begin{array}{cc}0,& c_d>\stackrel{\&OverBar;}{c_d}\\ \mathrm{\&alpha;}*(c_d-\stackrel{\&OverBar;}{c_d})/\stackrel{\&OverBar;}{|c_d-\stackrel{\&OverBar;}{c_d}|},& c_d<c_d\end{array}\right.$ (Formula 3); Among them, c _d is the cost of the segmentation block corresponding to the disparity d, and α is the weight coefficient. 8. the multiple soft constraint stereo matching method based on cost matrix according to claim 1, is characterized in that: this layer cost accumulation result described in step 7 is delivered to lower floor cost matrix, and its concrete realization comprises the following sub-steps: Step 7.1: Perform one-dimensional least square convolution on the cost value of each pixel within the parallax range; Step 7.2: According to the pyramid pixel correspondence, add the cost value of each pixel in this layer to the cost value of the corresponding disparity of the corresponding pixel in the lower layer. 9. The multiple soft-constrained stereo matching method based on cost matrix according to claim 1, characterized in that: the matching confidence degree described in step 9 is the ratio of the second minimum value of the matching cost and the minimum value, for only the matching confidence degree Pixels greater than a predetermined threshold can obtain parallax for the first time; according to the points that have obtained parallax as controls, the method of cost diffusion for unmatched points in the cost matrix is as follows: mark the matched points and unmatched points, and then determine each matched point The unmatched pixels in the window neighborhood correspond to the disparity cost of the matched point minus a certain penalty value, as shown in the following formula: $\cos t(q,d)=\left\{\begin{array}{c}\cos t(q,d)-P,\mathrm{D.}\left(p\right)=d\\ \cos t(q,d),\mathrm{D.}\left(p\right)\&NotEqual;d\end{array}\right.,q\&Element;N_d,\mathrm{D.}\left(q\right)=\mathrm{NULL}$ (style four); Among them, p represents the matched point, q represents the unmatched point, P represents the penalty value, and D(p) represents the value of the initial disparity map D at point p.