CN103714549A

CN103714549A - Stereo image object segmentation method based on rapid local matching

Info

Publication number: CN103714549A
Application number: CN201310745606.3A
Authority: CN
Inventors: 武港山; 夏春容; 居然
Original assignee: Nanjing University
Current assignee: Nanjing University
Priority date: 2013-12-30
Filing date: 2013-12-30
Publication date: 2014-04-09
Anticipated expiration: 2033-12-30
Also published as: CN103714549B

Abstract

Stereo image object segmentation method based on fast local stereo matching, to obtain reliable matching points on the image, and use Delaunay triangulation to perform interpolation to obtain parallax; then construct a graph, use the pixel as a vertex, and the connection line between the pixel and its eight neighbors as the edge of the graph , the weight of the edge is determined by the color and disparity information of the connected pixels; the Kruskal minimum spanning tree strategy is used to determine whether the area where the two connected pixels belong to the same segmented area according to the weight of the edge, if so, merge, otherwise remain unchanged; The obtained segmented area is judged whether it belongs to the object to be segmented, and the segmented area is taken out to obtain the final object. The method of the invention is fast and effective, and can effectively process parallax discontinuous regions such as object edges; multiple objects can be quickly segmented, and the invention has high time efficiency and good segmentation effect, and can meet the requirement of rapid automatic object segmentation.

Description

Stereo Image Object Segmentation Method Based on Fast Local Matching

技术领域technical field

本发明涉及双目立体视觉领域的立体匹配方法、对象分割方法，属于计算机视觉领域，主要应用是快速获取双目图像的视差信息，从而利用视差信息和图片本身的颜色信息进行对象分割，为一种基于快速局部匹配的立体图像对象分割方法。The invention relates to a stereo matching method and an object segmentation method in the field of binocular stereo vision, and belongs to the field of computer vision. A Stereo Image Object Segmentation Method Based on Fast Local Matching.

背景技术Background technique

立体视觉使得人们能够获取物体和场景中的深度信息，是后期应用包括3D重建和内容分析的基础。立体匹配是立体视觉的关键技术之一，它通过对两幅或者多幅图像进行像素配准来获取像素的深度信息。Stereo vision enables people to obtain depth information in objects and scenes, and is the basis for later applications including 3D reconstruction and content analysis. Stereo matching is one of the key technologies of stereo vision. It obtains the depth information of pixels by registering pixels of two or more images.

匹配精确度和运行时效性是立体匹配的两个核心因素，然而，前期的研究表明，这两个因素是相矛盾的。一般的，根据匹配策略，立体匹配方法可以分为两类：全局和局部立体匹配。全局立体匹配方法能够产生高精度的深度图，然而花费的时间较长。与此相反，局部立体匹配方法产生的深度图没有全局方法精度高，但是时效性高，能够满足实时或者接近实时的运用需求。所以，按照实际运用的需求，局部立体匹配方法值得我们进行深入的研究。Matching accuracy and runtime are two core factors of stereo matching, however, previous studies have shown that these two factors are contradictory. Generally, according to the matching strategy, stereo matching methods can be divided into two categories: global and local stereo matching. The global stereo matching method can generate a high-precision depth map, but it takes a long time. In contrast, the depth map produced by the local stereo matching method is not as accurate as the global method, but it is time-sensitive and can meet the real-time or near-real-time application requirements. Therefore, according to the actual application requirements, the local stereo matching method is worthy of our in-depth research.

视差插值广泛地运用于局部立体匹配中，这主要基于在局部平滑区域视差保持连续变化。对三角区域进行视差插值也在先前的研究中被采用。这些三角区域是通过对一些初始匹配点运用三角化形成的，假设三角区域内的视差保持连续变化，利用该三角区域形成的视差面进行插值是可行的。然而，如果该三角形出现在物体的边界或者形成它的顶点包含异常初始匹配点，利用该三角形进行插值得到的视差则是无效的。针对这些问题，本发明中的局部立体匹配方法给出了自己的解决方案，并且实验结果也印证了该方案的可行性。Disparity interpolation is widely used in local stereo matching, which is mainly based on the continuous change of disparity in locally smooth regions. Parallax interpolation for triangular regions was also employed in previous studies. These triangular areas are formed by applying triangulation to some initial matching points. Assuming that the disparity in the triangular area keeps changing continuously, it is feasible to use the disparity surface formed by the triangular area for interpolation. However, if the triangle appears on the boundary of the object or the vertices forming it contain abnormal initial matching points, the disparity obtained by interpolating with the triangle is invalid. Aiming at these problems, the local stereo matching method in the present invention provides its own solution, and the experimental results also confirm the feasibility of the solution.

对象分割，即图像分割，一直是计算机视觉的研究热点，它是大多数视觉问题(如对象识别)的基础。如何快速有效地对图像进行分割一直是该研究领域的重点与难点。目前分割效果较好的算法大多时间复杂度高，并且需要人工交互，分割的对象需要手动指定，这类算法的代表是基于图割的分割算法。与此相反，以均值漂移为代表的分割算法，时间效率高，并且不需要人工投入，但是它的分割效果大大不如图割算法，容易过分割，同一对象极易分割成多个区域。Object segmentation, i.e. image segmentation, has been a research hotspot in computer vision, and it is the basis of most vision problems such as object recognition. How to segment images quickly and effectively has always been the focus and difficulty of this research field. At present, most of the algorithms with good segmentation effects have high time complexity and require human interaction. The objects to be segmented need to be manually specified. The representative of this type of algorithm is the segmentation algorithm based on graph cuts. In contrast, the segmentation algorithm represented by mean shift has high time efficiency and does not require manual input, but its segmentation effect is much inferior to that of the graph cut algorithm, and it is easy to over-segment, and the same object is easily divided into multiple regions.

发明内容Contents of the invention

本发明一个方面提供了一种有效地基于扩展三角插值的局部立体匹配方法，该方法能够有效解决基于三角化的局部立体匹配算法在某些区域，如跨越物体边界的三角形或者由异常匹配点组成的三角形，插值不准确的问题。One aspect of the present invention provides an effective local stereo matching method based on extended triangulation interpolation, which can effectively solve the problem of local stereo matching algorithm based on triangulation in certain areas, such as triangles crossing object boundaries or composed of abnormal matching points. The triangle, the problem of inaccurate interpolation.

本发明的技术方案为：基于快速局部匹配的立体图像对象分割方法，包括以下步骤：The technical scheme of the present invention is: the stereo image object segmentation method based on fast local matching, comprises the following steps:

1)求取可靠匹配点：输入校验后的左图像，右图像对左图像进行均匀采样，对采样点运用自适应权重局部立体匹配方法在右图像中搜寻最佳匹配点，在计算像素点的匹配代价时采用sobel纹理特征与像素点的RGB颜色信息，然后对匹配结果进行后期校验，最后剩下的点作为可靠匹配点；1) Obtaining reliable matching points: input the verified left image, the right image uniformly samples the left image, uses the adaptive weight local stereo matching method to search for the best matching point in the right image, and calculates the pixel points The matching cost uses the sobel texture feature and the RGB color information of the pixel, and then checks the matching result in the later stage, and finally the remaining points are used as reliable matching points;

2)对可靠匹配点运用Delaunay三角化：将左图像求得的可靠匹配点组织成三角网格，对每个三角形进行可靠性判断，判断的依据是检查三角形与其临近的三角形是否共面，对三角形内的点，根据三角形的可靠性，利用三角形及与其共边的三个三角形进行插值求视差；对得到的视差图进行后期校验得到最终的视差图；2) Apply Delaunay triangulation to the reliable matching points: organize the reliable matching points obtained from the left image into triangular grids, and judge the reliability of each triangle. For the points in the triangle, according to the reliability of the triangle, use the triangle and the three triangles on the same side to interpolate to find the disparity; perform post-verification on the obtained disparity map to obtain the final disparity map;

3)对左图像构建图：左图像每个像素作为图的顶点，将像素与其八近邻的连线作为图的边，边的权值由相连像素的颜色和视差信息决定，视差信息由最终的视差图得到；3) Construct a graph for the left image: each pixel of the left image is used as the vertex of the graph, and the connection between the pixel and its eight neighbors is used as the edge of the graph, and the weight of the edge is determined by the color and disparity information of the connected pixels, and the disparity information is determined by the final The disparity map is obtained;

4)利用构建的图进行图像分割：采用Kruskal最小生成树策略，将边按照权值的大小由低到高排序，根据边的权值确定相连的两像素所在区域是否属于同一分割区域，判断的阈值根据区域自适应变化，依次处理完图中的所有边，则图片中所有像素都划归到相应的区域，得到分割区域；4) Use the constructed graph for image segmentation: use the Kruskal minimum spanning tree strategy to sort the edges from low to high according to their weights, and determine whether the areas where two connected pixels belong to the same segmented area according to the weights of the edges. The threshold value changes adaptively according to the area, and after processing all the edges in the graph in turn, all the pixels in the image are assigned to the corresponding area, and the segmented area is obtained;

5)提取出分割区域中感兴趣的对象：对得到的分割区域，按照视差大小、分割区域的紧凑度及分割区域所占整幅图片的比例，判断是否属于需要分割出来的对象，按照对象划分取出各对象的分割区域，完成对象分割。5) Extract the object of interest in the segmented area: For the obtained segmented area, according to the size of the parallax, the compactness of the segmented area, and the proportion of the segmented area to the entire picture, determine whether it belongs to the object that needs to be segmented, and divide it according to the object The segmentation area of each object is taken out, and the object segmentation is completed.

后期校验采用左右一致性检测。Later verification adopts left and right consistency detection.

所述步骤2)中，对每个三角形利用其临近的三角形进行可靠性判断具体为：In the step 2), the reliability judgment of each triangle using its adjacent triangles is specifically:

根据三角形的平面法向量的夹角θ判断三角形是否共面：According to the included angle θ of the plane normal vector of the triangle to determine whether the triangle is coplanar:

$θ θ = = {cos cos}^{- - 11} \frac{\overset{&RightArrow; &Right Arrow;}{{n no}_{{f f}_{11}}} \cdot \cdot \overset{&RightArrow; &Right Arrow;}{{n no}_{{f f}_{22}}}}{| | | | \overset{&RightArrow; &Right Arrow;}{{n no}_{{f f}_{11}}} | | | | \cdot &Center Dot; | | | | \overset{&RightArrow; &Right Arrow;}{{n no}_{{f f}_{22}}} | | | |} - - - - - - ((11))$

是两三角形f₁与f₂的法向量，如果夹角θ小于预先设置的阈值τ_f，则两三角形f₁与f₂认为位于同一平面，即共面；反之则位于不同平面；

is the normal vector of the two triangles f ₁ and f ₂ , if the included angle θ is smaller than the preset threshold τ _f , the two triangles f ₁ and f ₂ are considered to be on the same plane, that is, they are coplanar; otherwise, they are on different planes;

三角形f的可靠度γ(f)计算为：The reliability γ(f) of triangle f is calculated as:

$γ γ ((f f)) = = \frac{{t t}_{r r} + + {τ τ}_{r r}}{{t t}_{n no} + + {τ τ}_{r r}},, {t t}_{r r} \leq \leq {t t}_{n no} - - - - - - ((22))$

t_r是与三角形f共面的相邻三角形数目，t_n是与三角形f相邻三角形总数目，τ_r是对可靠度进行调整的参数。t _r is the number of adjacent triangles coplanar with triangle f, t _n is the total number of adjacent triangles with triangle f, τ _r is a parameter to adjust the reliability.

进一步的，步骤2)中根据三角形的可靠性求视差具体为：Further, in step 2), calculating the parallax according to the reliability of the triangle is specifically:

如果三角形f_p可靠，即其临近的三角形都与其共面，可靠度为1，则利用三角形决定的视差面对其内的点p求视差：If the triangle f _p is reliable, that is, its adjacent triangles are coplanar with it, and the reliability is 1, then use the parallax determined by the triangle to find the parallax for the point p inside it:

${d d}_{p p} = = {a a}_{{f f}_{p p}} {p p}_{x x} + + {b b}_{{f f}_{p p}} {p p}_{y the y} + + {c c}_{{f f}_{p p}} - - - - - - ((44))$

p_x与p_y是三角形内点p的坐标，

与是三角形f_p决定的视差平面；p _x and p _y are the coordinates of point p inside the triangle,

and is the parallax plane determined by the triangle f _p ;

如果三角形f_p不可靠，对其内的像素点采用贝叶斯模型求视差，以三角形f_p及其临近共边的三角形作为模型先验，同时增加一个参数化的平滑项作为模型先验，贝叶斯模型如下：If the triangle f _p is unreliable, use the Bayesian model to find the parallax for the pixels in it, use the triangle f _p and its adjacent triangles as the model prior, and add a parameterized smoothing item as the model prior, The Bayesian model is as follows:

P(D|I，F)∝P(I|D)P(F|D)P(D) (3)P(D|I，F)∝P(I|D)P(F|D)P(D) (3)

其中D代表视差，I代表图像，F代表三角形，P表示模型中的概率；where D stands for disparity, I stands for image, F stands for triangle, and P stands for probability in the model;

根据贝叶斯模型，求视差就是极大值式(3)的后验概率问题，极大值后验概率等价于极小值如下的局部能量函数：According to the Bayesian model, finding the disparity is the posterior probability problem of the maximum value formula (3), and the maximum value posterior probability is equivalent to the local energy function with the minimum value as follows:

E(p，f_p)=E_data(p，f_p)+λ_sE_smooth(p)+λ_fE_f(f_p) (5)E(p, f _p )=E _data (p, f _p )+λ _s E _smooth (p)+λ _f E _f (f _p ) (5)

E(p，f_p)表示所求的三角形f_p内点p的总匹配代价；E_data(p，f_p)是假设点p在三角面f_p上的匹配代价；E_smooth(p)是平滑项，用于描述点p和相邻点之间的关系，E_f(f_p)是视差面f_p不可靠的惩罚项，用于描述f_p的平面可靠性，λ_s和λ_f分别是平滑项和f_p可靠度的控制参数。E(p, f _p ) represents the total matching cost of the point p in the triangle f _p ; E _data (p, f _p ) is the matching cost of the hypothetical point p on the triangle f _p ; E _smooth (p) is The smoothing term is used to describe the relationship between a point p and its neighbors, E _f (f _p ) is a penalty term for the unreliability of the disparity surface f _p , and it is used to describe the plane reliability of f _p , λ _s and λ _f are respectively is the control parameter of the smoothing term and f _p reliability.

在步骤3)构建图时，基于假设：临近的颜色相似的像素应该位于同一对象内，临近的深度信息一致的两像素应该位于同一对象内，设定边的权值由相连两像素的颜色和视差共同决定，如果图中边相连的两像素都包含视差信息，则将视差和颜色按照7：3的比例确定边的权值，否则，只按照颜色信息确定权值。When constructing the graph in step 3), based on the assumption that adjacent pixels with similar colors should be located in the same object, and two adjacent pixels with consistent depth information should be located in the same object, the weight of the edge is set by the color and The parallax is jointly determined. If two pixels connected by an edge in the figure contain parallax information, the parallax and color are determined according to the ratio of 7:3 to determine the weight of the edge. Otherwise, only the color information is used to determine the weight.

作为优选方式，步骤4)中，第一次进行图像分割后，进行后期处理：结合视差信息，将临近的分割较细的属于同一视差面的分割区域进行合并，这里用于判断是否合并的分割细度根据图像分割需求确定。As a preferred mode, in step 4), after the image is segmented for the first time, post-processing is performed: combined with the disparity information, the adjacent segmented regions belonging to the same parallax plane with finer segmentation are merged, which is used here to determine whether to merge. The fineness is determined according to the image segmentation requirements.

步骤5)中需要分割出来的对象的判断原则为：The judgment principle of the object that needs to be segmented in step 5) is:

A)视差最大的区域：深度靠前，对象离镜头近，处于视觉显著位置；A) The area with the largest parallax: the depth is in the front, the object is close to the lens, and is in a visually significant position;

B)分割区域紧凑：即分割区域内像素数目与分割区域所在最小矩形区域面积的比值最小；B) The segmented area is compact: that is, the ratio of the number of pixels in the segmented area to the area of the smallest rectangular area where the segmented area is located is the smallest;

C)分割区域所占整个图像区域的比例适中：长宽比例符合预先设定的参数：C) The ratio of the segmented area to the entire image area is moderate: the aspect ratio conforms to the preset parameters:

$\frac{W \times H}{W_{all} \times H_{all}} > \frac{1}{250}, \frac{W \times H}{W_{all} \times H_{all}} < \frac{1}{4}, \frac{W}{H} < 5$ 以及 $\frac{H}{W} < 5$ $\frac{W \times h}{W_{all} \times h_{all}} > \frac{1}{250}, \frac{W \times h}{W_{all} \times h_{all}} < \frac{1}{4}, \frac{W}{h} < 5$ as well as $\frac{h}{W} < 5$

按照上述原则取出分割区域，得到分割对象；According to the above principle, the segmentation area is taken out to obtain the segmentation object;

其中，设q_num是属于分割区域内像素点总数，d_avg是分割区域内像素点的平均视差，w_min是分割区域最小横坐标值，w_max是最大横坐标值，h_min是分割区域最小纵坐标值，h_max是最大纵坐标值，则分割区域由公式(18)最小矩形确定，W和H分别是该最小矩形的长与宽，W_all与H_all是整幅图像的长与宽：Among them, let q _num be the total number of pixels belonging to the segmented area, d _avg be the average parallax of the pixels in the segmented area, w _min be the minimum abscissa value of the segmented area, w _max be the maximum abscissa value, and h _min be the minimum value of the segmented area The ordinate value, h _max is the maximum ordinate value, then the segmented area is determined by the minimum rectangle of formula (18), W and H are the length and width of the minimum rectangle respectively, and _Wall and _Hall are the length and width of the entire image :

W=w_max-w_min，H=h_max-h_min (18)W=w _max -w _min ，H=h _max -h _min (18)

对符合原则C)的分割区域，按照每个分割区域复杂度O从大到小排序，取出排在前面的n个对象，分割的复杂度为：For the segmented areas that meet the principle C), sort them according to the complexity O of each segmented area from large to small, and take out the top n objects. The complexity of the segmentation is:

$O o = = {d d}_{avg avg} \times \times \frac{{q q}_{num num}}{W W \times \times H h} - - - - - - ((1919))$

取出分割区域的最小矩形掩模，进行二值化空洞填充，然后将该掩模作用于原图得到分割出来的对象。Take out the minimum rectangular mask of the segmented area, perform binary hole filling, and then apply the mask to the original image to obtain the segmented object.

本发明与现有技术相比有如下优点Compared with the prior art, the present invention has the following advantages

本发明提出的局部立体匹配算法能有效解决现存基于三角插值的局部立体匹配算法在视差不连续区域插值不准确的问题，这些区域包括物体边界附近和由异常初始匹配点组成的三角区域。较之现在主流的全局立体匹配算法，该方法虽然在匹配精度上较低且得到的视差图不是稠密的，但是这并不影响后期在分割上面的运用。由于该方法采用的匹配特征是sobel边缘纹理，所以在对象与对象之间纹理特征丰富的地方视差匹配较准确，能够有效地指导对象分割，而对于纹理特征不明显的区域，虽然视差信息不准确或者缺失，依靠颜色信息也能够顺利地将同一对象的像素联系在一起。此外，本发明的局部立体匹配方法一个最大优势是时间效率高，更具有实用性。The local stereo matching algorithm proposed by the present invention can effectively solve the problem of inaccurate interpolation in discontinuous areas of disparity in existing local stereo matching algorithms based on triangular interpolation, and these areas include the vicinity of object boundaries and triangular areas composed of abnormal initial matching points. Compared with the current mainstream global stereo matching algorithm, although the matching accuracy of this method is lower and the obtained disparity map is not dense, this does not affect the later application of segmentation. Since the matching feature used in this method is the sobel edge texture, the parallax matching is more accurate in places with rich texture features between objects, which can effectively guide object segmentation. For areas with inconspicuous texture features, although the disparity information is not accurate Or missing, relying on color information can also successfully link the pixels of the same object together. In addition, one of the biggest advantages of the local stereo matching method of the present invention is that it has high time efficiency and is more practical.

本发明提出的基于颜色和深度的对象分割算法能够快速有效地分割出图像中的多个对象，与现存主流的基于图割的全局分割算法相比，虽然分割效果较差，但是分割过程中不需要人工交互，分割对象是多个而并非前后背景单一地进行分割，且时间效率更高；分割效果与均值漂移算法相比，分割时间效率更高，且分割结果非过分割。The object segmentation algorithm based on color and depth proposed by the present invention can quickly and effectively segment multiple objects in the image. Compared with the existing mainstream global segmentation algorithm based on graph cut, although the segmentation effect is poor, the Manual interaction is required, and the segmentation object is multiple rather than a single segmentation of the front and rear backgrounds, and the time efficiency is higher; the segmentation effect is more time-efficient than the mean shift algorithm, and the segmentation result is not over-segmented.

附图说明Description of drawings

图1为本发明的实施流程。Fig. 1 is the implementation process of the present invention.

图2为本发明的快速局部立体匹配方法的实施流程。Fig. 2 is an implementation flow of the fast local stereo matching method of the present invention.

图3为本发明的对象分割的实施流程。FIG. 3 is an implementation flow of object segmentation in the present invention.

图4为本发明对不可靠三角形内的点进行插值的模型。Fig. 4 is a model for interpolating points in an unreliable triangle according to the present invention.

具体实施方式Detailed ways

本发明提出了一种基于快速局部匹配的立体图像对象分割方法，包括以下步骤：The present invention proposes a stereoscopic image object segmentation method based on fast local matching, comprising the following steps:

1)输入校验后的左右双目立体图像，对左图像进行常规采样，采样的目的主要是为了保证算法的效率，左右图像都是原始输入，这里对左图像进行采样，和右图暂时还没关系。然后对采样点运用经典的匹配精度高的自适应权重局部立体匹配方法，在计算像素点的匹配代价时采用sobel纹理特征与像素点的RGB颜色信息，然后对匹配结果运用后期校验，如左右一致性检测，求出最终可靠的匹配点。1) Input the verified left and right binocular stereo images, and perform regular sampling on the left image. The purpose of sampling is mainly to ensure the efficiency of the algorithm. The left and right images are the original input. Here, the left image is sampled, and the right image is temporarily restored. It doesn't matter. Then apply the classic adaptive weight local stereo matching method with high matching accuracy to the sampling points, use the sobel texture feature and the RGB color information of the pixel when calculating the matching cost of the pixel, and then apply the post-check to the matching result, such as left and right Consistency detection, find the final reliable matching point.

2)对步骤1)求出的可靠匹配点运用Delaunay三角化，使得整幅图象被三角网格覆盖。对每个三角形，利用临近的三角形计算他的可靠度。如果三角形可靠，则利用该三角形决定的视差面对其内的像素点插值求视差；如果不可靠，则利用该三角形及其临近的三个共边三角形对其内的像素点插值求视差，同样对得到的视差图进行后期校验得到最终的视差图，该视差图可能是稀疏的。2) Apply Delaunay triangulation to the reliable matching points obtained in step 1), so that the whole image is covered by triangular mesh. For each triangle, calculate its reliability using neighboring triangles. If the triangle is reliable, use the parallax determined by the triangle to interpolate the pixel points in it to find the parallax; if it is unreliable, use the triangle and its adjacent three co-edge triangles to interpolate the pixel points in it to find the parallax, similarly The final disparity map is obtained by post-checking the obtained disparity map, and the disparity map may be sparse.

3)对左图像构建图，像素作为图的顶点，像素与其八近邻的连线作为图的边。边的权值由相连两像素的颜色和视差决定。3) Construct a graph for the left image, the pixel is used as the vertex of the graph, and the connection line between the pixel and its eight neighbors is used as the edge of the graph. The weight of an edge is determined by the color and disparity of two connected pixels.

4)采用Kruskal最小生成树策略，将边按照权值的大小由低到高排序，根据边的权值确定相连的两像素所在区域是否属于同一分割区域，判断的阈值是根据区域自适应变化的。依次处理完图中的所有边，则图片中所有像素都划归到相应的区域。初次分割容易过分割，后期处理中结合视差信息，将临近的分割较细的属于同一视差面的分割区域进行合并。4) Using the Kruskal minimum spanning tree strategy, the edges are sorted from low to high according to the weight value, and according to the weight value of the edge, it is determined whether the area where two connected pixels belong to the same segmented area, and the judgment threshold is adaptively changed according to the area . After processing all the edges in the graph in turn, all the pixels in the image are assigned to the corresponding regions. The initial segmentation is easy to over-segment, and the disparity information is combined in the post-processing to merge the adjacent segmentation regions with finer segmentation and belonging to the same disparity plane.

5)对得到的分割区域，按照视差大小、分割区域的紧凑度以及分割区域所占整幅图片的比例大小判断是否属于需要得到的对象，取出分割区域，得到最终分割出来的对象。5) For the obtained segmented area, judge whether it belongs to the object to be obtained according to the size of the parallax, the compactness of the segmented area, and the ratio of the segmented area to the entire picture, and take out the segmented area to obtain the final segmented object.

如图1为本发明的实施流程图，首先对校验后的左右双目立体图像利用现存的自适应权重的方法求可靠匹配点。为了保证算法的效率，先对左图像进行均匀采样，本方法所采用的采样窗口是3×3。对于采样点用经典的自适应权重的方法在右图像中搜寻最佳匹配点。在聚集匹配代价时按照局部窗口内像素自身信息和几何距离确定的权重进行代价累计相加。给定像素点p，对于局部窗口内的任一像素点q，q的权重是由p、q之间的颜色差异c_pq和欧几里得距离g_pq确定的，本发明的说明中，三角形内的点也是像素点：Fig. 1 is the implementation flowchart of the present invention, at first utilize the method for existing self-adaptive weight to obtain reliable matching point to the left and right binocular stereoscopic images after verification. In order to ensure the efficiency of the algorithm, the left image is uniformly sampled first, and the sampling window used in this method is 3×3. For the sampling points, the classic adaptive weight method is used to search for the best matching point in the right image. When aggregating the matching cost, the cost is accumulated and added according to the weight determined by the pixel's own information and geometric distance in the local window. Given a pixel point p, for any pixel point q in the local window, the weight of q is determined by the color difference c _pq and the Euclidean distance g _pq between p and q, in the description of the present invention, the triangle The points inside are also pixels:

$ω ω ((p p,, q q)) = = exp exp ((- - ((\frac{{c c}_{pq pq}}{{σ σ}_{c c}} + + \frac{{g g}_{pq pq}}{{σ σ}_{g g}})))) - - - - - - ((66))$

σ_c与σ_g是用于控制颜色和欧几里得距离的参数，每个像素的聚集代价C′(p，d)如下：σ _c and σ _g are parameters used to control the color and Euclidean distance, and the aggregation cost C′(p, d) of each pixel is as follows:

${C C}^{' '} ((p p,, d d)) = = {Σ Σ}_{q q &Element; &Element; {Ω Ω}_{p p}} ω ω ((p p,, q q)) C C ((q q,, d d)) - - - - - - ((77))$

Ω_p是像素点p的局部窗口，C(q，d)是左图像中的点q(x，y)与右图像中的点q_d(x-d，y)的匹配代价，本发明的匹配代价是像素点的纹理特征sobel差异与像素点的RGB颜色信息。然后采用赢者通吃(winner-take-all)的策略找到匹配代价最小的视差。Ω _p is the local window of pixel p, C(q, d) is the matching cost of point q(x, y) in the left image and point q _d (xd, y) in the right image, and the matching cost of the present invention is the texture feature sobel difference of the pixel and the RGB color information of the pixel. Then adopt the winner-take-all strategy to find the disparity with the least matching cost.

D(p)=arg min_d C′(p，d) (8)D(p)=arg min _d C'(p,d) (8)

D(p)是像素点p的视差，即左图像中的像素点p(x，y)与右图像中的像素点p(x-D(p)，y)匹配，对左图像中的所有采样点求取视差后，运用后期处理，如左右一致性检验，除去一些不太可靠的点，最终剩下的点就是需要的可靠匹配点。使用Delaunay三角化将这些可靠点组织成三角形，左图像所有像素点都被三角网格覆盖，如图5所示。D(p) is the disparity of pixel p, that is, the pixel point p(x, y) in the left image matches the pixel point p(x-D(p), y) in the right image, and all sampling points in the left image After calculating the disparity, use post-processing, such as left-right consistency check, to remove some unreliable points, and finally the remaining points are the required reliable matching points. Use Delaunay triangulation to organize these reliable points into triangles, and all pixels in the left image are covered by triangle meshes, as shown in Figure 5.

对每个三角形，利用临近的三角形计算他的可靠度。大多数情况下，同一对象内的像素点，它们的视差应该位于同一平面内，因而位于同一对象的三角形也应该具有相似的视差平面，如果三角形跨越物体的边界或者组成三角形的点是不可靠的，则该三角形决定的视差面不适合对其内的点进行插值求视差，它是不可靠的。因而本发明提出利用其临近的三角形判断每个三角形的可靠性，可靠的适合插值的三角形应该与其周围的三角形在相似的视差面上。判断三角形是否位于同一平面可以根据他们的平面法向量的夹角：For each triangle, calculate its reliability using neighboring triangles. In most cases, the disparity of pixels in the same object should be in the same plane, so the triangles in the same object should also have similar disparity planes. If the triangle crosses the boundary of the object or the points forming the triangle are unreliable , then the parallax surface determined by the triangle is not suitable for interpolating the points in it to find the parallax, and it is unreliable. Therefore, the present invention proposes to use its adjacent triangles to judge the reliability of each triangle, and a reliable triangle suitable for interpolation should be on a similar parallax plane as its surrounding triangles. Judging whether the triangles are on the same plane can be based on the angle between their plane normal vectors:

$θ θ = = {cos cos}^{- - 11} \frac{\overset{&RightArrow; &Right Arrow;}{{n no}_{{f f}_{11}}} \cdot \cdot \overset{&RightArrow; &Right Arrow;}{{n no}_{{f f}_{22}}}}{| | | | \overset{&RightArrow; &Right Arrow;}{{n no}_{{f f}_{11}}} | | | | \cdot \cdot | | | | \overset{&RightArrow; &Right Arrow;}{{n no}_{{f f}_{22}}} | | | |} - - - - - - ((11))$

是两三角形f₁与f₂的法向量，如果夹角θ小于某一提前设置的阈值τ_f，则两三角形f₁与f₂近似认为位于同一平面；反之则位于不同平面。如果共面的三角形越多该三角形的可靠性也就越高，反之则可靠性越低。三角形f的可靠度γ(f)计算为：

is the normal vector of the two triangles f ₁ and f _2. If the included angle θ is smaller than a threshold τ _f set in advance, the two triangles f ₁ and f ₂ are approximately considered to be on the same plane; otherwise, they are on different planes. If there are more coplanar triangles, the reliability of the triangle is higher, otherwise, the reliability is lower. The reliability γ(f) of triangle f is calculated as:

t_r是与三角形f共面的相邻三角形数目，t_n是相邻三角形总数目，τ_r是对可靠度进行调整的参数。t _r is the number of adjacent triangles coplanar with triangle f, t _n is the total number of adjacent triangles, τ _r is a parameter to adjust the reliability.

A)对于可靠的三角形γ(f_p)=1，即其临近的三角形都与其共面，可靠度为1，利用该三角形决定的视差平面对其内的点插值求视差：A) For a reliable triangle γ(f _p )=1, that is, its adjacent triangles are all coplanar with it, and the reliability is 1, use the parallax plane determined by the triangle to interpolate the points in it to find the parallax:

p_x与p_y是三角形f_p内点p的坐标，

与

是三角形f_p决定的视差平面；p _x and p _y are the coordinates of point p inside the triangle f _p ,

and

is the parallax plane determined by the triangle f _p ;

B)对于不可靠的三角形γ(f_p)＜1，利用该三角形及其共边临近的三个三角形对其内的像素点p插值求视差。对其内的像素点采用贝叶斯模型求视差，如图4，以三角形f_p及其临近共边的三角形作为模型先验，同时增加一个参数化的平滑项作为模型先验，贝叶斯模型如下：B) For an unreliable triangle γ(f _p )<1, use the triangle and its three adjacent triangles on the same side to interpolate the pixel point p to obtain the parallax. Use the Bayesian model to find the parallax for the pixels in it, as shown in Figure 4, take the triangle f _p and its adjacent triangles as the model prior, and add a parameterized smoothing item as the model prior, Bayesian The model is as follows:

P(D|I，F)∝P(I|D)P(F|D)P(D) (3)P(D|I，F)∝P(I|D)P(F|D)P(D) (3)

D是需要计算的视差，I是图像本身，F是三角形决定的视差面，也即三角形自身，P是贝叶斯模型的概率。根据贝叶斯模型求视差就是极大值后验概率的问题，极大值后验概率等价于极小值如下的局部能量函数：D is the parallax that needs to be calculated, I is the image itself, F is the parallax surface determined by the triangle, that is, the triangle itself, and P is the probability of the Bayesian model. Finding the parallax according to the Bayesian model is the problem of the maximum posterior probability, which is equivalent to the local energy function with the minimum value as follows:

E(p，f_p)是点p的总匹配代价；E_data(p，f_p)是假设点p在三角形f_p决定的视差面上的匹配代价；E_smooth(p)是平滑项，如果点p与相邻的像素点不在同一视差面会有相应的惩罚值，E_smooth(p)用于描述点p和相邻点之间的关系，E_f(f_p)是视差面f_p不可靠的惩罚项，用于描述f_p的平面可靠性，λ_s和λ_f分别是平滑项和f_p可靠度的控制参数。E(p, f _p ) is the total matching cost of point p; E _data (p, f _p ) is the matching cost of hypothetical point p on the parallax surface determined by triangle f _p ; E _smooth (p) is a smooth item, if Point p and adjacent pixels are not on the same parallax surface, there will be a corresponding penalty value, E _smooth (p) is used to describe the relationship between point p and adjacent points, E _f (f _p ) is the parallax surface f _p is unreliable The penalty term of is used to describe the plane reliability of f _p , and λ _s and λ _f are the control parameters of the smoothing term and the reliability of f _p respectively.

本发明中，所记载的三角形、视差面和三角面三者是指同一个面，只是在从不同角度说到这个面时，名称描述上有些差异。例如，从Delaunay三角化的角度描述时，称为三角形，从视差的角度描述时称为视差面，从平面的角度描述时，称为三角面。In the present invention, the triangle, parallax plane and triangular plane mentioned refer to the same plane, but when referring to this plane from different angles, there are some differences in name description. For example, when described from the perspective of Delaunay triangulation, it is called a triangle, when described from the perspective of parallax, it is called a parallax surface, and when described from the perspective of a plane, it is called a triangular surface.

同样对得到的视差图进行后期校验得到最终的视差图，该视差图可能是稀疏的。Also, a post-check is performed on the obtained disparity map to obtain a final disparity map, which may be sparse.

一般情况下，假设三角形内视差是连续变化的，对于三角形内的点采用三角形确定的视差面进行插值可以确定视差，如公式(4)。然而这种简单的插值对那些跨越物体边界或者由初始异常匹配点组成的三角形是不适用的。根据大多数现实生活中的场景，在不可靠三角形内点的视差极有可能在其近邻可靠三角形的视差面上，所以采用临近的视差面可以有效地对不可靠三角形内的点进行视差插值。临近视差面的最佳数量是由采样密度和图像的具体细节决定。此外，时间代价也需要考虑在内，方法的计算复杂度与临近视差面的数量呈正相关关系。再者，临近视差面离参考点越远，其对该点的影响越小，所以本发明最终将临近共边的3个三角形f_n1，f_n2，f_n3作为候选的临近视差面，加上中间三角形f的视差面，最终用于插值模型的候选视差面由四个组成：In general, assuming that the parallax in the triangle changes continuously, the parallax can be determined by interpolating the points in the triangle using the parallax surface determined by the triangle, as shown in formula (4). However, this simple interpolation is not suitable for those triangles that straddle object boundaries or consist of initial anomalous matching points. According to most real-life scenarios, the disparity of a point in an unreliable triangle is likely to be on the disparity surface of its neighboring reliable triangle, so using the adjacent disparity surface can effectively perform disparity interpolation on points in an unreliable triangle. The optimal number of adjacent parallax planes is determined by the sampling density and the specific details of the image. In addition, the time cost also needs to be considered, and the computational complexity of the method is positively correlated with the number of adjacent disparity surfaces. Furthermore, the farther the adjacent parallax plane is from the reference point, the smaller its influence on the point, so the present invention finally uses three adjacent triangles f _n1 , f _n2 , and f _n3 as candidate adjacent parallax planes, plus The parallax surface of the middle triangle f, the final candidate parallax surface for the interpolation model consists of four:

${Θ Θ}_{{f f}_{p p}} = = {{f f,, {f f}_{n no 11},, {f f}_{n no 22},, {f f}_{n no 33}}} - - - - - - ((99))$

能量函数(5)中，E_data(p，f_p)是假设像素点p在视差面f_p上的匹配代价；E_smooth(p)是4近邻已匹配像素之间的平滑项。考虑到算法效率，只采用一次处理，最开始已匹配的像素可能比较少，但是这并不会对结果造成较大偏差，因为整个平滑项在能量函数中是非硬性限制。In the energy function (5), E _data (p, f _p ) is the matching cost of the hypothetical pixel point p on the disparity surface f _p ; E _smooth (p) is the smoothing item between the 4 adjacent matched pixels. Considering the efficiency of the algorithm, only one processing is used, and there may be fewer matched pixels at the beginning, but this will not cause a large deviation to the result, because the entire smoothing term is a non-hard limit in the energy function.

${E E.}_{smooth smooth} ((p p)) = = {Σ Σ}_{q q &Element; &Element; {N N}_{44}} Δ Δ {d d}_{pq pq} - - - - - - ((1010))$

${Δd Δd}_{pq pq} = = \{\begin{matrix} 11,, & abs abs (({d d}_{p p} - - {d d}_{q q})) > > 11 \\ 00,, & otherwise otherwise \end{matrix} - - - - - - ((1111))$

E_f(f_p)是平面f_p不可靠性惩罚项，按照公式(2)可以确定f_p的可靠度。E _f (f _p ) is the unreliability penalty item of plane f _p , and the reliability of f _p can be determined according to formula (2).

${E E.}_{f f} (({f f}_{p p})) = = \frac{11}{γ γ (({f f}_{p p}))} - - - - - - ((1212))$

极小化能量函数(5)，根据赢者通吃策略找到最佳视差面Minimize the energy function (5), and find the best parallax surface according to the winner-take-all strategy

$f f ((p p)) = = {arg arg min min}_{{f f}_{p p} &Element; &Element; {Θ Θ}_{{f f}_{p p}}} E E. ((p p,, {f f}_{p p})) - - - - - - ((1313))$

找到最佳视差面，根据公式(4)，计算出点p的视差。Find the best parallax surface, and calculate the parallax of point p according to formula (4).

对生成的视差图运用后期处理，如左右一致性检验，则得到可靠的稀疏视差图。Applying post-processing to the generated disparity maps, such as left-right consistency checks, results in reliable sparse disparity maps.

步骤3)对左图像构建图，如图3所示，像素点作为图的顶点，每个像素点与其八近邻的连线作为边，边的权值大小根据相连的两像素的颜色和深度信息共同决定：Step 3) Construct a graph to the left image, as shown in Figure 3, the pixel point is used as the vertex of the graph, and the connection line between each pixel point and its eight neighbors is used as an edge, and the weight of the edge is based on the color and depth information of the connected two pixels decided together:

如果相连两像素都有可靠的视差值，则颜色和视差按照3：7的比例确定权值；反之，则只采用颜色信息作为权值。If two connected pixels have reliable disparity values, the color and disparity determine the weight according to the ratio of 3:7; otherwise, only the color information is used as the weight.

设p_r，p_g，p_b是像素点p的RGB三通道颜色值，q_r，p_g，p_b是像素点p的RGB三通道颜色值，颜色差异W_c如公式(14)Suppose p _r , p _g , p _b are the RGB three-channel color values of pixel p, q _r , p _g , p _b are the RGB three-channel color values of pixel p, and the color difference W _c is as in formula (14)

${W W}_{c c} = = \sqrt{{(({p p}_{r r} - - {q q}_{r r}))}^{22} + + {(({p p}_{g g} - - {q q}_{g g}))}^{22} + + {(({p p}_{b b} - - {q q}_{b b}))}^{22}} - - - - - - ((1414))$

p_d，q_d是像素点p，q的视差值，视差差异W_d如公式(15)p _d , q _d is the parallax value of pixel p, q, and the parallax difference W _d is as formula (15)

W_d=|p_d-q_d| (15)W _d =|p _d -q _d | (15)

边的权值w如公式(16)The weight w of the edge is as formula (16)

$W W = = \{\begin{matrix} 0.7 0.7 \times \times {W W}_{d d} + + 0.3 0.3 \times \times {W W}_{c c},, & {p p}_{d d} &NotEqual; &NotEqual; 00 and and {q q}_{d d} &NotEqual; &NotEqual; 00 \\ 0.5 0.5 \times \times {W W}_{c c},, & otherwise otherwise \end{matrix} - - - - - - ((1616))$

p与q是同一边相连的两像素，如果p与q都含有视差，确定权值时视差差异占0.7，颜色差异占0.3；否则取颜色差异的0.5确定权值，这里之所以取0.5，主要是为了在不考虑视差的情况下使颜色对分割的影响不至于太强。p and q are two pixels connected on the same side. If both p and q contain parallax, the parallax difference accounts for 0.7 when determining the weight, and the color difference accounts for 0.3; otherwise, 0.5 of the color difference is used to determine the weight. It is to make the effect of color on the segmentation not too strong without considering the parallax.

构建好了图，采用Kruskal’s最小生成树策略进行图像分割，分割的复杂度为O(mlogm)，具体的分割算法如下：After constructing the graph, Kruskal's minimum spanning tree strategy is used for image segmentation. The complexity of segmentation is O(mlogm). The specific segmentation algorithm is as follows:

输入图G=(V，E)，输出分割区域：S=(C₁，...，C_r)Input graph G=(V, E), output segmentation area: S=(C ₁ ,...,C _r )

对边E按照非降的顺序进行排序，排序后为π=(o₁，...，o_m)Sort the edge E in non-descending order, after sorting, it is π=(o ₁ ,...,o _m )

a.初始的分割结果是S₀，每个像素是一个分割区域a. The initial segmentation result is S ₀ , each pixel is a segmented area

b.对每条边q=1，...，m重复步骤cb. Repeat step c for each edge q=1,...,m

c.在前一次的分割结果基础上S_q-1，进行下一次分割S_q，取第q条边o_q=(v_i，v_j).假如按照S_q-1的分割结果，v_i和v_j在不同的分割区域，并且边的权值w(o_q)小于两分割区域的阈值，则将v_i和v_j所在的两分割区域合并。调整合并后的区域阈值。否则S_q=S_q-1。c. On the basis of the previous segmentation result S _q-1 , perform the next segmentation S _q , and take the qth side o _q =(v _i , v _j ). If according to the segmentation result of S _q-1 , v _i and v _j are in different segmentation areas, and the edge weight w(o _q ) is less than the threshold of the two segmentation areas, then merge the two segmentation areas where v _i and v _j are located. Adjust the merged region threshold. Otherwise S _q =S _q-1 .

d.当处理完所有边时返回d. Return when all edges are processed

在分割过程中，区域的阈值是自适应变化的，主要是为了保证分割结果既不会过分割也不会分割得很粗糙。During the segmentation process, the threshold of the region is adaptively changed, mainly to ensure that the segmentation result is neither over-segmented nor very rough.

T(C)=Int(C)+τ(C)τ(C)=k／|C| (17)T(C)=Int(C)+τ(C)τ(C)=k／|C| (17)

T(C)是分割区域的阈值，Int(C)是分割区域内相连像素的最大权值，τ(C)是自适应变化的值。k是提前设置的阈值，它一般用于控制分割区域的大小，在整个分割过程中是固定的。|C|是分割区域的大小，即像素的数目，它是自动变化的。T(C) is the threshold of the segmented area, Int(C) is the maximum weight of connected pixels in the segmented area, τ(C) is the value of adaptive change. k is a threshold set in advance, which is generally used to control the size of the segmented region and is fixed throughout the segmentation process. |C| is the size of the segmented area, that is, the number of pixels, which changes automatically.

依次处理完图中的所有边，则图片中所有像素都划归到相应的区域，得到分割区域。After processing all the edges in the graph in sequence, all the pixels in the image are assigned to the corresponding regions, and the segmented regions are obtained.

虽然设定了区域阈值，初次分割还是比较容易过分割，作为优选方式，在后期处理中结合视差信息，将临近的分割较细的属于同一视差面的分割区域进行合并，这里用于判断是否合并的分割细度根据图像分割需求确定。这样在初次分割时可以分割细一点，然后通过后期处理进行调整，达到合适的分割结果。Although the region threshold is set, the initial segmentation is still relatively easy to over-segment. As a preferred method, in the post-processing, the disparity information is combined to merge the adjacent segmentation regions that belong to the same disparity surface with finer segmentation. Here, it is used to judge whether to merge The segmentation fineness of is determined according to the image segmentation requirements. In this way, the segmentation can be finer during the initial segmentation, and then adjusted through post-processing to achieve a suitable segmentation result.

步骤5)中，对分割后得到的区域，需要取出其中合适的对象。q_num是属于分割区域内像素点总数；d_avg是分割区域内像素点的平均视差；w_min是分割区域最小横坐标值，w_max是最大横坐标值；h_min是分割区域最小纵坐标值，h_max是最大纵坐标值。则取出的分割区域可以由公式(18)最小矩形确定，W和H分别是该最小矩形的长与宽，W_all与H_all是整幅图像的长与宽。In step 5), for the regions obtained after segmentation, appropriate objects need to be taken out. q _num is the total number of pixels belonging to the segmented area; d _avg is the average parallax of pixels in the segmented area; w _min is the minimum abscissa value of the segmented area, w _max is the maximum abscissa value; h _min is the minimum ordinate value of the segmented area , h _max is the maximum ordinate value. Then the extracted segmented area can be determined by the smallest rectangle in formula (18), W and H are the length and width of the smallest rectangle respectively, and Wall and _Hall are the length and _width of the entire image.

W=w_max-w_min，H=h_max-h_min (18)W=w _max -w _min ，H=h _max -h _min (18)

取出需要的对象原则是The principle of taking out the required objects is

a.视差最大：深度最靠前，对象离镜头最近，处于视觉显著位置。a. The largest parallax: the depth is the most front, the object is closest to the lens, and is in a visually prominent position.

b.分割区域紧凑：即分割区域内像素数目与分割区域所在最小矩形区域面积的比值最小；b. The segmented area is compact: that is, the ratio of the number of pixels in the segmented area to the area of the smallest rectangular area where the segmented area is located is the smallest;

c.分割区域所占整个图像区域的比例比较适中：长宽比例合适，满足事先设定的阈值。c. The ratio of the segmented area to the entire image area is relatively moderate: the aspect ratio is appropriate and meets the preset threshold.

符合原则a、b的分割区域直接取出就可以，如果分割区域所占整幅图像的比例比较合适，即符合原则c，既不太小也不太大且长宽比例合适，即

以及

则按照每个区域O的大小由大到小排序，取出排在前面的n个对象，具体n的大小可以自由指定，n个对象就是n个分割区域。分割的复杂度为：The segmented areas that meet the principles a and b can be taken out directly. If the proportion of the segmented area to the entire image is relatively appropriate, that is, it meets the principle c, and it is neither too small nor too large and the aspect ratio is appropriate, that is

as well as

Sort according to the size of each area O from large to small, and take out the top n objects. The specific size of n can be freely specified, and n objects are n divided areas. The complexity of the split is:

本发明提出的立体匹配算法快速有效，能够有效处理物体边缘等视差不连续区域；分割算法能够快速地分割出多个对象，本发明时间效率高，分割效果好，能够满足快速自动对象分割的需求。The stereo matching algorithm proposed by the present invention is fast and effective, and can effectively process discontinuous areas of parallax such as object edges; the segmentation algorithm can quickly segment multiple objects, and the present invention has high time efficiency and good segmentation effect, and can meet the needs of fast automatic object segmentation .

Claims

1. the stereo-picture Object Segmentation method based on quick local matching, is characterized in that comprising the following steps:

1) ask for reliable matching point: the left image after input validation, right image carries out uniform sampling to left image, to sampled point, use adaptive weighting sectional perspective matching process to search optimal match point in right image, when the coupling cost of calculating pixel point, adopt the RGB colouring information of sobel textural characteristics and pixel, then matching result is carried out to later stage verification, finally remaining point is as reliable matching point;

2) reliable matching point is used to Delaunay trigonometric ratio: the reliable matching point that left image is tried to achieve is organized into triangle gridding, each triangle is carried out to reliability judgement, judgement according to being to check that whether the triangle that triangle closes on it coplanar, point in diabolo, according to leg-of-mutton reliability, utilize three triangles on triangle and limit common with it to carry out interpolation and ask parallax; The disparity map obtaining is carried out to later stage verification and obtain final disparity map;

3) to left picture construction figure: each pixel of left image is as the summit of figure, the limit using pixel and its eight neighbour's line as figure, the weights on limit determine by color and the parallax information of the pixel that is connected, parallax information is obtained by final disparity map;

4) utilizing the figure building to carry out image cuts apart: adopt Kruskal minimum spanning tree strategy, sorted from low to high according to the size of weights in limit, according to the weights on limit, determine whether the two pixel regions that are connected belong to same cut zone, the threshold value of judgement changes according to region adaptivity, handle successively all limits in figure, in picture, all pixels all incorporate corresponding region into, obtain cut zone;

5) extract interested object in cut zone: to the cut zone obtaining, according to parallax size, the compactness of cut zone and the ratio of the shared picture in its entirety of cut zone, judge whether to belong to the object that need to split, according to object, divide the cut zone of taking out each object, complete Object Segmentation.

2. the stereo-picture Object Segmentation method based on quick local matching according to claim 1, is characterized in that described later stage verification adopts left and right consistency detection.

3. the stereo-picture Object Segmentation method based on quick local matching according to claim 1, is characterized in that described step 2) in, to each triangle, utilize its triangle closing on to carry out reliability judgement and be specially:

According to the angle theta of leg-of-mutton planar process vector, judge that whether triangle is coplanar:

θ = \cos^{- 1} \frac{\overset{&RightArrow;}{n_{f_{1}}} \cdot \overset{&RightArrow;}{n_{f_{2}}}}{| | \overset{&RightArrow;}{n_{f_{1}}} | | \cdot | | \overset{&RightArrow;}{n_{f_{2}}} | |} - - - (1)

two triangle f ₁with f ₂normal vector, if angle theta is less than the threshold tau setting in advance _f, two triangle f ₁with f ₂think and be positioned at same plane, coplanar; Otherwise be positioned at Different Plane;

The fiduciary level γ of triangle f (f) is calculated as:

γ (f) = \frac{t_{r} + τ_{r}}{t_{n} + τ_{r}}, t_{r} \leq t_{n} - - - (2)

T _rthe adjacent triangle number coplanar with triangle f, t _ntriangle total number adjacent with triangle f, τ _rit is the parameter that fiduciary level is adjusted.

4. the stereo-picture Object Segmentation method based on quick local matching according to claim 3, is characterized in that step 2) in, according to leg-of-mutton reliability, ask parallax to be specially:

If triangle f _preliably, its triangle closing on is all coplanar with it, and fiduciary level is 1, utilizes parallax that triangle determines to ask parallax in the face of the some p in it:

d_{p} = a_{f_{p}} p_{x} + b_{f_{p}} p_{y} + c_{f_{p}} - - - (4)

P _xwith p _ythe coordinate of some p in triangle,

with

triangle f _pthe parallax plane determining;

If triangle f _punreliable, to the pixel in it, adopt Bayesian model to ask parallax, with triangle f _pand the triangle that closes on common limit is as model priori, increases a parameterized level and smooth item as model priori simultaneously, Bayesian model is as follows:

P(D|I，F)∝P(I|D)P(F|D)P(D) (3)

Wherein D represents parallax, I representative image, and F represents triangle, P represents the probability in model;

According to Bayesian model, asking parallax is exactly the posterior probability problem of maximum value formula (3), and maximum value posterior probability is equivalent to the local energy function that minimal value is following:

E(p，f _p)=E _data(p，f _p)+λ _sE _smooth(p)+λ _fE _f(f _p) (5)

E (p, f _p) represent required triangle f _ptotal coupling cost of interior some p; E _data(p, f _p) be that postulated point p is at triangular facet f _pon coupling cost; E _smooth(p) it is level and smooth, for describing the relation between a p and consecutive point, E _f(f _p) be parallax face f _pinsecure penalty term, for describing f _pplane reliability, λ _sand λ _frespectively level and smooth and f _pthe control parameter of fiduciary level.

5. the stereo-picture Object Segmentation method based on quick local matching according to claim 1, it is characterized in that in step 3) during design of graphics, based on hypothesis: the pixel of the color similarity closing on should be positioned at same target, the two consistent pixels of depth information of closing on be positioned at same target, the weights on setting limit are determined jointly by color and the parallax of two pixels that are connected, if two pixels that in figure, limit is connected all comprise parallax information, by parallax and color according to the weights on the ratio-dependent limit of 7:3, otherwise, only according to colouring information, determine weights.

6. the stereo-picture Object Segmentation method based on quick local matching according to claim 1, it is characterized in that step 4) in, carry out for the first time after image cuts apart, carry out post-processed: in conjunction with parallax information, the thinner cut zone that belongs to same parallax face of cutting apart of closing on is merged, for the fineness of cutting apart that judges whether to merge, according to image, cut apart Location of requirement here.

7. the stereo-picture Object Segmentation method based on quick local matching according to claim 1, is characterized in that step 5) in need the judgment principle of the object that splits to be:

A) region of parallax maximum: the degree of depth is forward, object is close to camera lens, in the remarkable position of vision;

B) cut zone is compact: in cut zone, the ratio of number of pixels and cut zone place minimum rectangular area area is minimum;

C) ratio of the shared whole image-region of cut zone is moderate: Aspect Ratio meets predefined parameter:

\frac{W \times H}{W_{all} \times H_{all}} > \frac{1}{250}, \frac{W \times H}{W_{all} \times H_{all}} < \frac{1}{4}, \frac{W}{H} < 5

And

\frac{H}{W} < 5

According to mentioned above principle, take out cut zone, obtain cutting object;

Wherein, establish q _numto belong to pixel sum in cut zone, d _avgthe mean parallax of pixel in cut zone, w _minthe minimum abscissa value of cut zone, w _maxmaximum abscissa value, h _minthe minimum ordinate value of cut zone, h _maxbe maximum ordinate value, cut zone is determined by formula (18) minimum rectangle, and W and H are respectively the length of this minimum rectangle and wide, W _allwith H _allthe length of entire image and wide:

W=w _max-w _min，H=h _max-h _min (18)

To with the C that is in conformity with the principle) cut zone, according to each cut zone complexity O, sort from big to small, take out and to come n object above, the complexity of cutting apart is:

O = d_{avg} \times \frac{q_{num}}{W \times H} - - - (19)

Take out the minimum rectangle mask of cut zone, carry out binaryzation cavity and fill, then this mask is acted on to the object that former figure obtains splitting.