WO2018098891A1

WO2018098891A1 - Stereo matching method and system

Info

Publication number: WO2018098891A1
Application number: PCT/CN2017/070639
Authority: WO
Inventors: 唐荣富; 余勤力; 周剑; 龙学军; 徐一丹
Original assignee: 成都通甲优博科技有限责任公司
Priority date: 2016-11-30
Filing date: 2017-01-09
Publication date: 2018-06-07
Also published as: CN106780442A; CN106780442B

Abstract

A stereo matching method and system. The method comprises: extracting feature points of left and right images and performing feature point matching on the feature points to determine support points (S100); constructing a Delaunay triangle according to the support points, the Delaunay triangle comprising prior probabilities of all pixel point parallaxes within the triangle, and the minimum support distances between pixel points and the support points (S110); calculating parallax conditional probabilities and parallax confidence levels of pixel points in the left image using a parallax calculation method (S120); and calculating, according to the Delaunay triangle, the parallax conditional probabilities, and the parallax confidence levels, an optimal posterior parallax using a Bayesian method (S130). The method implements rapid matching to obtain a high precision parallax image, and is particularly suitable for a mobile platform or application fields having high requirements on real-timeliness.

Description

Stereo matching method and system

The present application claims priority to Chinese Patent Application No. PCT Application No. No. No. No. No. No. No. No. No. No. No. No. No. No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No .

Technical field

The present invention relates to the field of computer vision technology, and in particular, to a stereo matching method and system.

Background technique

Stereo dense matching refers to the correspondence between the features of the same spatial physical point in different images according to the calculation of the selected features and the correspondence between the features. Stereo matching is an important hotspot and difficulty in computer vision research. It is one of the key technologies in many applications such as robotics, medicine and artificial intelligence. In recent years, with the development of mobile platforms, the accuracy and real-time requirements of the stereo matching method have been continuously improved. The stereo matching algorithm is decomposed into four steps: matching cost calculation, matching cost aggregation, disparity calculation, and parallax refinement. According to different constraints, the stereo matching algorithm can be divided into local matching algorithm and global matching algorithm.

The global stereo matching algorithm mainly estimates the parallax through the global optimization theory method, establishes the global energy function, and then obtains the optimal disparity value by minimizing the global energy function. In general, the global matching algorithm obtains higher accuracy than the local algorithm, but it is computationally intensive and time consuming, and is not suitable for real-time applications. The main algorithms are graph cuts, belief propagation, semi-global matching, dynamic programing, and so on. At present, a research direction of stereo matching is a machine learning method using a convolutional neural network, which can obtain an accuracy equal to or higher than that of the classical global algorithm.

The local matching algorithm mainly uses the local optimization method to estimate the disparity value. Like the global stereo matching algorithm, the parallax estimation is also performed by the energy minimization method. The difference is that in the energy function, the local matching algorithm has only data items, but no smoothing. item. Because the local matching algorithm is sensitive to changes in illumination intensity and contrast, when the image has repeated texture features, weak texture and severe occlusion, the probability of false matching is high. Commonly used local matching algorithms mainly include SAD (sum of absolute differences algorithm, CT (census transform) algorithm, ASW (adaptive support) Weight) algorithm, ELAS (efficient large area stereo matching) algorithm, IDR (iterative dense refinement) algorithm, and the like. The SAD algorithm calculates the absolute value of the corresponding pixel difference of the local window. The CT algorithm first transforms the window region, and then calculates the matching cost according to the Hamming distance metric. The SAD and CT methods are simple and fast to implement, but with low accuracy. According to the principle of biological vision, the ASW algorithm changes the single weight of the SAD algorithm, introduces the idea of adaptive weight, and obtains a high matching precision, but the amount of computation caused by the adaptive weight is very large. The IDR algorithm uses a two-pass approach to simplify the implementation of ASW and adds an iterative improvement method to achieve higher accuracy. The IDR algorithm structure is very conducive to parallel processing. It can get high computational efficiency after optimization under CUDA architecture. However, IDR algorithm also has two main disadvantages: it runs slowly under non-CUDA architecture, and the memory overhead is very large. The ELAS algorithm adopts a completely different idea from the above method: it first uses the sobel operator to obtain the strong texture support points of the image; then uses the support point geometry to perform Delaunay triangulation on the image to obtain the parallax plane estimation of the pixel; The sobel operator measure calculates the matching cost and uses the weighting method to obtain the optimal estimate of the disparity. ELAS algorithm is one of the fastest running stereo matching algorithms, and its accuracy is also very high, suitable for real-time applications. The main shortcomings of the ELAS algorithm are: the algorithm structure is not conducive to parallelization, and there are some cases where pixel points cannot be calculated.

Therefore, how to improve the operating efficiency under the premise of ensuring the accuracy of the algorithm is still a difficult point in the application of the stereo matching technology in the mobile platform.

Summary of the invention

The object of the present invention is to provide a stereo matching method and system, which realizes fast matching to obtain a high-precision parallax map, and is particularly suitable for a mobile platform or an application field with high real-time requirements.

To solve the above technical problem, the present invention provides a stereo matching method, including:

Extracting feature points of the left and right maps, and performing feature point matching on the feature points to determine support points;

Constructing a Delaunay triangle according to the support point; wherein the Delaunay triangle includes a prior probability of parallax of all pixel points in the triangle and a minimum support distance of the pixel point and the support point;

Using the parallax calculation method to calculate the parallax condition probability and the parallax confidence level of the pixel points in the left image;

According to the Delaunay triangle, the disparity condition probability and the parallax confidence level are calculated by using a Bayesian principle to obtain an optimal a posteriori parallax.

Optionally, extract feature points of the left and right images, and perform feature point matching on the feature points to determine support points, including:

The feature points are extracted from the left and right graphs by using the FAST operator, and the feature description is performed by using theRIEF;

The feature points are matched by the polar line constraint and the Hamming distance in the feature description, and the feature points with successful matching are used as the support points.

Optionally, constructing a Delaunay triangle according to the support point includes:

Using the divide and conquer method to perform Delaunay triangulation on the set of support points on the left graph;

Using formula

Calculating a minimum support distance m between the pixel point and the support point of the left image;

Gaussian model

Calculating a prior probability P(dn|S) of the parallax of the pixel of the left picture;

Where D _p,i is the Euclidean distance of the corresponding pixel point and the support point of its Delaunay triangle, σ _p =mσ, σ is the variance, m is the constant parameter; dp is the disparity estimate determined by the support point, and dp=au _p +bv _p +c, the parameters a, b, c are obtained by fitting three support point planes, dn is the support point disparity value, and u _p and v _p are the abscissa and ordinate of the pixel point, respectively.

Optionally, the parallax calculation method is used to calculate the parallax condition probability and the parallax confidence level of the pixel points in the left image, including:

The parallax condition probability and the parallax confidence level of the pixel points in the left picture are calculated by using an improved census transform stereo matching algorithm.

Optionally, according to the Delaunay triangle, the disparity condition probability and the disparity confidence level, using a Bayesian principle to calculate an optimal a posteriori parallax, including:

Using formula

Calculate the optimal posterior parallax d ^* ;

among them,

For the parallax posterior probability, f(dn) is the Hamming distance of the improved census transform stereo matching algorithm, f _m (d) is the confidence level function of the conditional parallax, S is the support point, and O is based on some local stereo matching operator. Matching cost, α is the weight parameter.

Optionally, the stereo matching method further includes:

The left and right consistency detection method is used to detect the mismatched points in the disparity map.

Optionally, after detecting the mismatched point in the disparity map by using the left and right consistency detection method, the method further includes:

According to the WTA policy, the disparity values of the error matching points are replaced by the disparity values on the left and right sides of the mismatching point.

The invention also provides a stereo matching system comprising:

a support point determining module, configured to extract feature points of the left and right images, and perform feature point matching on the feature points to determine a support point;

a Delaunay triangle building module, configured to construct a Delaunay triangle according to the support point; wherein the Delaunay triangle includes a prior probability of parallax of all pixel points in the triangle and a minimum support distance of the pixel point and the support point;

a probability calculation module, configured to calculate a parallax condition probability and a parallax confidence level of a pixel point in the left image by using a parallax calculation method;

The a posteriori parallax calculation module is configured to calculate an optimal a posteriori parallax by using a Bayesian principle according to the Delaunay triangle, the parallax condition probability and the parallax confidence level.

Optionally, the Delaunay triangle building module includes:

a splitting unit for performing Delaunay triangulation on the set of support points of the left figure by using the divide and conquer method;

Distance calculation unit for using formulas

Prior probability calculation unit for utilizing a Gaussian model

Optionally, the a posteriori parallax calculation module specifically uses a formula

Calculating a module that obtains an optimal a posteriori parallax d ^* ;

among them,

The stereo matching method provided by the present invention comprises: extracting feature points of the left and right images, and performing feature point matching on the feature points to determine support points; constructing a Delaunay triangle according to the support points; wherein the Delaunay triangle includes the first parallax of all the pixels in the triangle Probability and the minimum support distance between the pixel and the support point; use the disparity calculation method to calculate the parallax condition probability and the parallax confidence level of the pixel in the left picture; according to the Delaunay triangle, the disparity condition probability and the disparity confidence level, calculate by Bayesian principle The optimal a posteriori parallax is obtained. The method uses fast, high-precision and adaptive stereo matching method to achieve fast matching and obtain high-precision parallax map, which is especially suitable for mobile platforms or applications with high real-time requirements. A stereo matching system is provided, which has the above-mentioned beneficial effects and will not be described herein.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is an embodiment of the present invention, and those skilled in the art can obtain other drawings according to the provided drawings without any creative work.

FIG. 1 is a flowchart of a stereo matching method according to an embodiment of the present invention;

FIG. 2 is a structural block diagram of a stereo matching system according to an embodiment of the present invention.

detailed description

The core of the invention is to provide a stereo matching method and system, which realizes fast matching to obtain a high-precision parallax map, and is particularly suitable for a mobile platform or an application field with high real-time requirements.

The technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. It is a partial embodiment of the invention, and not all of the embodiments. Based on the present invention All other embodiments obtained by those skilled in the art without creative efforts are within the scope of the present invention.

In the description of the following embodiments, the general term of stereo matching is adopted, and the left image is used as a reference image, and the left and right images have been completed to perform camera calibration and stereo rectification.

Please refer to FIG. 1. FIG. 1 is a flowchart of a stereo matching method according to an embodiment of the present invention; the method may include:

S100: Extract feature points of the left and right images, and perform feature point matching on the feature points to determine support points;

Specifically, the step is mainly for obtaining a support point, and the embodiment does not limit the extraction and matching algorithm of the specific feature point. In order to complete the parallax map quickly and accurately, it is possible to select a correlation algorithm with high calculation speed and high precision. Furthermore, in order to make the stereo matching algorithm work well on mobile platforms, the selected algorithm should also have the characteristics of relatively simple computational logic. For example, when feature point matching is performed, feature point matching can be quickly performed by using polar line constraints and feature point descriptors, and the matching feature points are called support points. Here, the polar constraint reduces the matching search from two-dimensional to one-dimensional, greatly simplifying the complexity and increasing the calculation speed. Suitable for mobile platforms or applications with high real-time requirements.

Preferably, extracting the feature points of the left and right images and performing feature point matching on the feature points to determine the support points may include:

Specifically, the feature points are extracted from the left and right graphs by using the FAST (features from accelerated segment test) operator and characterized by a BRIEF Fiction (BRIEF).

Among them, FAST feature point detection is recognized as a fast and effective feature point extraction method. FAST feature extraction mainly includes three steps: segmentation test on pixels on a fixed radius (usually selected as three pixels) circle, and removing a large number of non-feature candidate points through logic test; based on classification of corner feature detection, utilization The ID3 classifier determines whether the candidate points are corner features according to the 16 features; the non-maximum suppression is used to verify the corner feature. The BRIEF descriptor is a gray-scale calculation that randomly takes a pair of points around the feature points to directly obtain a binary feature description vector. The BRIEF descriptor has two distinct advantages: the descriptor requires less bytes and the memory overhead is small; the Hamming measure is very fast. fast.

Specifically, feature point matching is quickly performed using polar line constraints and feature point distances, and the matching feature points are called support points.

Among them, the polar line constraint reduces the matching search from two-dimensional to one-dimensional, greatly simplifying the complexity. Considering the limitation of the disparity space (Disp), the matching points on the right picture of the feature points on the left picture can only be located between one cell on the corresponding outer pole line. Therefore, in this embodiment, a WTA (winner takes all) strategy can be used to select a point with the smallest matching cost as a matching point in the parallax space, and the disparity D _L (p) of the p point on the corresponding left image is

Where: d(d∈Disp) represents a possible disparity in the disparity space Disp, which is generally an integer between 0 and the maximum disparity d _max ; H(.) represents a Hamming distance corresponding to the left and right pixel BRIEF descriptors.

S110. Construct a Delaunay triangle according to the support point; wherein the Delaunay triangle includes a prior probability of parallax of all pixel points in the triangle and a minimum support distance of the pixel point and the support point;

Specifically, the purpose of Delaunay triangulation on the left image is to divide the image into triangular meshes that cover the entire image plane and are connected to each other, and describe the disparity map as a series of triangular regions with the same or similar disparity values, triangles. The mesh can reflect the topological connection between the pixel and its neighboring pixels. For the parallax smoothing region, the triangle division should be large enough to reduce the ambiguity of the matching while ensuring the edge details. In the continuous parallax area, the vertex density and number should be as small as possible to speed up the matching. In the parallax discontinuous area, in order to correctly describe the shape of the object, the number of vertices should be sufficient to better ensure the accuracy of subsequent disparity map matching.

Among them, Delaunay triangulation has the following advantages: good structure, simple data structure, small data redundancy, high storage efficiency, and consistent with irregular ground features, which can represent linear features and can adapt to data of various distribution densities.

This embodiment does not limit the Delaunay triangulation algorithm, and the user can The choice of the result is selected, and the hardware level of the calculation platform, the requirements for the accuracy of the result and the calculation speed can be considered in the algorithm selection. The Delaunay triangulation algorithm may include a random increment method, a triangulation method, and a divide and conquer method. Among them, the random incremental method is simple and efficient, and takes up less memory, but its time complexity is high. The triangulation growth method is less efficient because of its relatively low efficiency. The efficiency of the divide and conquer method is the highest, and after splitting. The triangular patches are smoother while maintaining the edge features of the object. Therefore, it is preferable that the present embodiment can perform Delaunay triangulation on the set of support points using the divide and conquer method. In this embodiment, the Delaunay triangle provides information such as the prior probability of the parallax of all the pixels in the triangle and the minimum support distance of the pixel and the support point; specifically, constructing the Delaunay triangle according to the support point may include :

Using formula

Gaussian model

Where D _p,i is the Euclidean distance of the corresponding pixel point and the support point of its Delaunay triangle, σ _p =mσ, σ is the variance, m is a constant parameter (can be set and modified according to the empirical value); dp is The disparity estimate determined by the support point, and dp=au _p +bv _p +c, the parameters a, b, c are obtained by fitting three support point planes, dn is the support point disparity value, u _p and v _p are respectively The abscissa and ordinate of the pixel.

Specifically, after performing the Delaunay triangulation on the support point set of the left graph by using the divide and conquer method, since all the support point disparity values are known, it is possible to estimate any pixel point included in the plane determined by each triangle. The initial value of the parallax dp, and the minimum support distance m (minimum support distance) of the pixel and the adjacent three support points is obtained by geometric relationship.

S120. Calculate a parallax condition probability and a parallax confidence level of the pixel point in the left figure by using a parallax calculation method;

Specifically, this step is mainly for calculating the disparity condition probability and the confidence level of any pixel point in the left figure, and does not limit the specific parallax calculation method. The improved census transform stereo matching algorithm has the advantages of simple structure and fast calculation speed. Therefore, Preferably, the parallax condition probability and the parallax confidence level of the pixel points in the left picture are calculated by using an improved census transform stereo matching algorithm.

Specifically, the census transform is a non-parametric local transform. The downside is that the result is too dependent on the center pixel. Therefore, the present embodiment adopts a modified census transform based on the neighborhood information for stereo matching algorithm. The improved census transform stereo matching algorithm is proposed to improve the census transform stereo matching algorithm based on the correlation information of the traditional census transform in the case of parallax discontinuous region and noise interference. The two-information is used to represent the gray-scale difference between the pixel and the central pixel, the pixel and the neighborhood gray-scale mean. The census transform is improved, and the initial matching cost is obtained by Hamming distance calculation; the parallel layered weighted cost aggregation improves the matching precision, and Reduce the cost of aggregation calculations.

The improved census transform stereo matching algorithm makes the representation of the central pixel more precise; the information representation of the transformed image in the parallax discontinuous region is more abundant; and the influence of noise on the matching quality is reduced. The test shows that the algorithm is simple in structure, low in complexity, high in robustness, and effectively improves matching accuracy. This step uses the Hamming distance to represent the parallax conditional probability P(O|dn):

Where f(dn) is the Hamming distance of the improved census transform stereo matching algorithm, and f _m (d) is the confidence level function of the conditional disparity. From a statistical point of view, the parallax conditional probability characterizes the confidence level of the disparity dn.

S130. Calculate an optimal a posteriori parallax by using a Bayesian principle according to the Delaunay triangle, the disparity condition probability, and the disparity confidence level. Optimal posterior parallax is the optimal posterior estimate of parallax

Specifically, according to the prior probability, the minimum support distance, the disparity condition probability and the parallax confidence level in the Delaunay triangle, the Bayesian principle is utilized.

The optimal posterior disparity is obtained, which is the best disparity of the pixel. According to the parallax prior probability model and the conditional probability model, the Bayesian parameter estimation model is obtained: P(dn|S, O) ∝ P(dn|S)*P(O|dn), and the following results are obtained:

among them,

For the parallax posterior probability, f(dn) is the Hamming distance of the improved census transform stereo matching algorithm, f _m (d) is the confidence level function of the conditional parallax, S is the support point, and O is based on some local stereo matching operator. Matching cost, α is the weight parameter, and dn is the parallax.

It should be particularly noted that the present invention provides a Bayesian stereo matching method for optimal disparity estimation, in which the algorithm of support point extraction and conditional probability is replaceable, and the corresponding model parameters are adjusted.

Based on the foregoing technical solution, the stereo matching method provided by the embodiment of the present invention can determine the weight parameters of the prior probability and the conditional probability according to the confidence level of the conditional parallax, the geometric topological relationship between the pixel point and the support point, and obtain a more accurate The parallax posterior estimate. Compared with the ELAS algorithm experience curing weight parameter in the prior art, the method makes full use of the information contained in the a priori parallax and the conditional parallax, and the model is more reasonable. The parameters are adaptive. In this method, the prior probability of the parameter model and the weight parameter of the conditional probability are adaptively determined according to the confidence level and the geometric topological relationship. In fact, the parametric model only needs to determine the empirical parameters α and σ (which can be determined experimentally), and σ _p and P(O|dn) are adaptively varied and more flexible. The parameter model is simple in form and efficient in operation. That is to say, the method utilizes a fast, high-precision, adaptive stereo matching method to achieve fast matching to obtain a high-precision parallax map, which is particularly suitable for mobile platforms or applications requiring high real-time performance.

Based on the foregoing technical solution, in order to improve the accuracy of the stereo matching, the method may further include:

Specifically, the mismatching point is detected using the left and right consistency detection method. Using the disparity maps of the left and right images obtained in the above embodiment, if the pixel points in the right disparity map are shifted to the left and the disparity values are the same After the unit pixel is the same as the value of the corresponding pixel in the left disparity map, it is a reliable matching point, otherwise it is a mismatching point. After detecting the mismatch point, the method may further include:

Specifically, for the calculated mismatch point, the WTA strategy can be replaced by the parallax values on the left and right sides. Improve the accuracy of stereo matching.

Further, at this time, there may still be a small amount of noise in the disparity map, and further denoising is needed, and the fast median filtering can be used to filter out. Finally, the parallax of sub-pixel precision is obtained by interpolation optimization, so that the disparity map is more complete and correct.

Based on the above technical solution, the stereo matching method provided by the embodiment of the present invention uses the FAST feature extraction operator and the BRIEF description operator to construct the Bayesian prior probability model, which improves the efficiency and density of the support point. To make the prior probability more accurate; when constructing the Bayesian conditional probability model, the improved CT algorithm (ie, the improved census transform algorithm) is more fast and accurate; more importantly, the Bayesian parameter estimation model of the method is used. The geometric topology of a priori parallax and the confidence level of conditional probability are fully considered. It has the characteristics of parameter adaptability, simple form and high efficiency.

The stereo matching system provided by the embodiment of the present invention is described below. The stereo matching system described below and the stereo matching method described above can refer to each other.

Please refer to FIG. 2. FIG. 2 is a structural block diagram of a stereo matching system according to an embodiment of the present invention; the system may include:

The support point determining module 100 is configured to extract feature points of the left and right images, and perform feature point matching on the feature points to determine a support point;

a Delaunay triangle building block 200, configured to construct a Delaunay triangle according to the support point; wherein the Delaunay triangle includes a prior probability of parallax of all pixel points in the triangle and a minimum support distance of the pixel point and the support point;

The probability calculation module 300 is configured to calculate a parallax condition probability and a parallax confidence level of the pixel points in the left image by using a parallax calculation method;

The a posteriori parallax calculation module 400 is configured to calculate an optimal a posteriori parallax by using a Bayesian principle according to the Delaunay triangle, the parallax condition probability and the parallax confidence level.

Based on the above embodiment, the Delaunay triangle building module 200 may include:

Distance calculation unit for using formulas

Prior probability calculation unit for utilizing a Gaussian model

Based on the above embodiment, the a posteriori parallax calculation module 400 specifically uses a formula

Calculating a module that obtains an optimal a posteriori parallax d ^* ;

among them,

Based on any of the above embodiments, the system may further include:

The consistency detection module is configured to detect the mismatched point in the disparity map by using the left and right consistency detection method.

Based on any of the above embodiments, the system may further include:

And a replacement module, configured to replace, by using a disparity value of the left and right sides of the mismatching point, a disparity value of the error matching point according to a WTA policy.

Based on any of the above embodiments, the system may further include:

The denoising module is used to filter out using fast median filtering, and finally the parallax of sub-pixel precision is obtained by interpolation optimization, so that the disparity map is more complete and correct.

Based on the above technical solution, the stereo matching system provided by the embodiment of the present invention obtains the parallax optimal estimation by using the idea of Bayesian maximum a posteriori estimation. The system first uses the support point to quickly match to obtain parallax Prior probability (prior probability), where prior probability is related to support point parallax, pixel point geometry, minimum distance; conditional probability is calculated by improved census transform stereo matching algorithm, where conditional probability and matching cost, confidence level Correlation; Finally, the posterior probability is obtained based on the prior probability and the conditional probability, and the optimal estimate of the disparity is obtained by maximizing the posterior probability. In particular, the parameters in the Bayesian model used by the system are adaptive.

The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on differences from other embodiments, and the same similar parts between the various embodiments may be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

A person skilled in the art will further appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software or a combination of both, in order to clearly illustrate the hardware and software. Interchangeability, the composition and steps of the various examples have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.

The steps of a method or algorithm described in connection with the embodiments disclosed herein can be implemented directly in hardware, a software module executed by a processor, or a combination of both. The software module can be placed in random access memory (RAM), memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or technical field. Any other form of storage medium known.

The stereo matching method and system provided by the present invention are described in detail above. The principles and embodiments of the present invention have been described herein with reference to specific examples, and the description of the above embodiments is only to assist in understanding the method of the present invention and its core idea. It should be noted that those skilled in the art can make various modifications and changes to the present invention without departing from the spirit and scope of the invention.

Claims

A stereo matching method, comprising:

Extracting feature points of the left and right maps, and performing feature point matching on the feature points to determine support points;

Constructing a Delaunay triangle according to the support point; wherein the Delaunay triangle includes a prior probability of parallax of all pixel points in the triangle and a minimum support distance of the pixel point and the support point;

Using the parallax calculation method to calculate the parallax condition probability and the parallax confidence level of the pixel points in the left image;

According to the Delaunay triangle, the disparity condition probability and the parallax confidence level are calculated by using a Bayesian principle to obtain an optimal a posteriori parallax.
The stereo matching method according to claim 1, wherein extracting feature points of the left and right images and performing feature point matching on the feature points to determine support points comprises:

The feature points are extracted from the left and right graphs by using the FAST operator, and the feature description is performed by using theRIEF;

The feature points are matched by the polar line constraint and the Hamming distance in the feature description, and the feature points with successful matching are used as the support points.
The stereo matching method according to claim 2, wherein constructing a Delaunay triangle according to the support point comprises:

Using the divide and conquer method to perform Delaunay triangulation on the set of support points on the left graph;

Using formula
Calculating a minimum support distance m between the pixel point and the support point of the left image;

Gaussian model
Calculating a prior probability P(dn|S) of the parallax of the pixel of the left picture;

Where D p,i is the Euclidean distance of the corresponding pixel point and the support point of its Delaunay triangle, σ p =mσ, σ is the variance, m is the constant parameter; dp is the disparity estimate determined by the support point, and dp=au p +bv p +c, the parameters a, b, c are obtained by fitting three support point planes, dn is the support point disparity value, and u p and v p are the abscissa and ordinate of the pixel point, respectively.
The stereo matching method according to claim 3, wherein the parallax condition probability and the parallax confidence level of the pixel points in the left image are calculated by using a parallax calculation method, including:

The parallax condition probability and the parallax confidence level of the pixel points in the left picture are calculated by using an improved census transform stereo matching algorithm.
The stereo matching method according to claim 4, wherein the parallax conditional probability and the parallax confidence level are calculated according to the Delaunay triangle, and the optimal a posteriori parallax is calculated by using a Bayesian principle, including:

Using formula
Calculate the optimal posterior parallax d * ;

among them,
For the parallax posterior probability, f(dn) is the Hamming distance of the improved census transform stereo matching algorithm, f m (d) is the confidence level function of the conditional parallax, S is the support point, and O is based on some local stereo matching operator. Matching cost, α is the weight parameter.
The stereo matching method according to any one of claims 1 to 5, further comprising:

The left and right consistency detection method is used to detect the mismatched points in the disparity map.
The stereo matching method according to claim 6, wherein after detecting the mismatching point in the disparity map by using the left and right consistency detecting method, the method further comprises:

According to the WTA policy, the disparity values of the error matching points are replaced by the disparity values on the left and right sides of the mismatching point.
A stereo matching system, comprising:

a support point determining module, configured to extract feature points of the left and right images, and perform feature point matching on the feature points to determine a support point;

a Delaunay triangle building module, configured to construct a Delaunay triangle according to the support point; wherein the Delaunay triangle includes a prior probability of parallax of all pixel points in the triangle and a minimum support distance of the pixel point and the support point;

a probability calculation module, configured to calculate a parallax condition probability and a parallax confidence level of a pixel point in the left image by using a parallax calculation method;

The a posteriori parallax calculation module is configured to calculate an optimal a posteriori parallax by using a Bayesian principle according to the Delaunay triangle, the parallax condition probability and the parallax confidence level.
The stereo matching method according to claim 8, wherein the Delaunay triangle building module comprises:

a splitting unit for performing Delaunay triangulation on the set of support points of the left figure by using the divide and conquer method;

Distance calculation unit for using formulas
Calculating a minimum support distance m between the pixel point and the support point of the left image;

Prior probability calculation unit for utilizing a Gaussian model
Calculating a prior probability P(dn|S) of the parallax of the pixel of the left picture;

Where D p,i is the Euclidean distance of the corresponding pixel point and the support point of its Delaunay triangle, σ p =mσ, σ is the variance, m is the constant parameter; dp is the disparity estimate determined by the support point, and dp=au p +bv p +c, the parameters a, b, c are obtained by fitting three support point planes, dn is the support point disparity value, and u p and v p are the abscissa and ordinate of the pixel point, respectively.
The stereo matching method according to claim 8, wherein the a posteriori parallax calculation module is specifically configured to calculate an optimal a posteriori parallax d * by using a formula d * =argminP(dn|S, O);

among them,
For the parallax posterior probability, f(dn) is the Hamming distance of the improved census transform stereo matching algorithm, f m (d) is the confidence level function of the conditional parallax, S is the support point, and O is based on some local stereo matching operator. Matching cost, α is the weight parameter.