CN103714549A - Stereo image object segmentation method based on rapid local matching - Google Patents

Abstract

Provided is a stereo image object segmentation method based on rapid local stereo matching. The stereo image object segmentation method based on rapid local stereo matching comprises the steps of obtaining a reliable matching point on an image, and using Delaunay triangulation to carry out interpolation so as to obtain the parallax; building a graph, using a pixel as a vertex, and using connecting lines formed between the pixel and eight-site neighborhood pixels of the pixel as edges of the graph, wherein the weight of each edge is determined by the colors of the connected pixels and parallax information; determining whether the areas where every two connected pixels are located belong to the same segmentation area or not through a Kruskal minimum spanning tree strategy according to the weights of the edges, if yes, combining the areas, and if not, keeping the areas unchanged; judging whether the obtained segmentation area belongs to an object needing to be segmented out or not, taking out the segmentation area, and obtaining the final object. The stereo image object segmentation method based on rapid local stereo matching is rapid and effective, areas with the discontinuous parallax, such as edges of articles can be effectively processed, a plurality of objects can be rapidly segmented out, time efficiency is high, the segmentation effect is good, and the requirement for automatic and rapid object segmentation can be met.

Description

Stereo-picture Object Segmentation method based on quick local matching

Technical field

The present invention relates to solid matching method, the Object Segmentation method in binocular stereo vision field, belong to computer vision field, main application is the parallax information of quick obtaining binocular image, thereby utilizing the colouring information of parallax information and picture itself to carry out Object Segmentation, is a kind of stereo-picture Object Segmentation method based on quick local matching.

Background technology

Stereoscopic vision makes people can obtain the depth information in object and scene, is that later stage application comprises that 3D rebuilds and the basis of content analysis.Stereo matching is one of gordian technique of stereoscopic vision, and it is by carrying out to two width or multiple image the depth information that pixel matching obtains pixel.

Coupling degree of accuracy and operation ageing be two central factors of Stereo matching, yet, early stage research show, these two factors are conflicting.General, according to matching strategy, solid matching method can be divided into two classes: overall and local Stereo matching.Overall situation solid matching method can produce high accuracy depth figure, yet the time of cost is longer.In contrast, the depth map that sectional perspective matching process produces does not have global approach precision high, but ageing height, can meet in real time or approach real-time utilization demand.So according to the demand of practice, sectional perspective matching process is worth us to carry out deep research.

Parallax interpolation applies in sectional perspective coupling widely, and this is mainly based on keeping changing continuously at local smoothing method regional disparity.Parallax interpolation is carried out in Delta Region to be also used in previous research.These Delta Regions, by using trigonometric ratio to form to some initial matching points, suppose that the parallax in Delta Region keeps changing continuously, and it is feasible that the parallax face that utilizes this Delta Region to form carries out interpolation.Yet if the summit that this triangle appears at the border of object or forms it comprises abnormal initial matching point, it is invalid utilizing this triangle to carry out the parallax that interpolation obtains.For these problems, the sectional perspective matching process in the present invention has provided the solution of oneself, and experimental result has also been confirmed the feasibility of this scheme.

Object Segmentation, image is cut apart, and is the study hotspot of computer vision always, and it is the basis of most of visual problems (as object identification).How to Image Segmentation Using, to be fast and effeciently emphasis and the difficult point of this research field always.The most time complexity of the good algorithm of segmentation effect is high at present, and needs man-machine interactively, and the object needs of cutting apart are manually specified, and the representative of this class algorithm is the partitioning algorithm cutting based on figure.In contrast, the partitioning algorithm that the average drifting of take is representative, time efficiency is high, and does not need artificial input, but its segmentation effect is much worse than greatly figure, cuts algorithm, easily over-segmentation, same target is very easily divided into a plurality of regions.

Summary of the invention

One aspect of the invention provides a kind of sectional perspective matching process based on extended triangular interpolation effectively, the method can effectively solve sectional perspective matching algorithm based on trigonometric ratio in some region, as cross over the triangle of object boundary or the triangle being formed by Exceptional point, the inaccurate problem of interpolation.

Technical scheme of the present invention is: the stereo-picture Object Segmentation method based on quick local matching, comprises the following steps:

1) ask for reliable matching point: the left image after input validation, right image carries out uniform sampling to left image, to sampled point, use adaptive weighting sectional perspective matching process to search optimal match point in right image, when the coupling cost of calculating pixel point, adopt the RGB colouring information of sobel textural characteristics and pixel, then matching result is carried out to later stage verification, finally remaining point is as reliable matching point;

2) reliable matching point is used to Delaunay trigonometric ratio: the reliable matching point that left image is tried to achieve is organized into triangle gridding, each triangle is carried out to reliability judgement, judgement according to being to check that whether the triangle that triangle closes on it coplanar, point in diabolo, according to leg-of-mutton reliability, utilize three triangles on triangle and limit common with it to carry out interpolation and ask parallax; The disparity map obtaining is carried out to later stage verification and obtain final disparity map;

3) to left picture construction figure: each pixel of left image is as the summit of figure, the limit using pixel and its eight neighbour's line as figure, the weights on limit determine by color and the parallax information of the pixel that is connected, parallax information is obtained by final disparity map;

4) utilizing the figure building to carry out image cuts apart: adopt Kruskal minimum spanning tree strategy, sorted from low to high according to the size of weights in limit, according to the weights on limit, determine whether the two pixel regions that are connected belong to same cut zone, the threshold value of judgement changes according to region adaptivity, handle successively all limits in figure, in picture, all pixels all incorporate corresponding region into, obtain cut zone;

5) extract interested object in cut zone: to the cut zone obtaining, according to parallax size, the compactness of cut zone and the ratio of the shared picture in its entirety of cut zone, judge whether to belong to the object that need to split, according to object, divide the cut zone of taking out each object, complete Object Segmentation.

Later stage verification adopts left and right consistency detection.

Described step 2), in, to each triangle, utilize its triangle closing on to carry out reliability judgement and be specially:

According to the angle theta of leg-of-mutton planar process vector, judge that whether triangle is coplanar:

$\mathrm{\θ}={\cos }^{-1}\frac{\stackrel{\→}{n_{f_1}}\·\stackrel{\→}{n_{f_2}}}{\left|\right|\stackrel{\→}{n_{f_1}}\left|\right|\·\left|\right|\stackrel{\→}{n_{f_2}}\left|\right|}---\left(1\right)$

two triangle f ₁with f ₂normal vector, if angle theta is less than the threshold tau setting in advance _f, two triangle f ₁with f ₂think and be positioned at same plane, coplanar; Otherwise be positioned at Different Plane;

The fiduciary level γ of triangle f (f) is calculated as:

$\mathrm{\γ}\left(f\right)=\frac{t_r+{\mathrm{\τ}}_r}{t_n+{\mathrm{\τ}}_r},t_r\≤t_n---\left(2\right)$

T _rthe adjacent triangle number coplanar with triangle f, t _ntriangle total number adjacent with triangle f, τ _rit is the parameter that fiduciary level is adjusted.

Further, step 2), according to leg-of-mutton reliability, ask parallax to be specially:

If triangle f _preliably, its triangle closing on is all coplanar with it, and fiduciary level is 1, utilizes parallax that triangle determines to ask parallax in the face of the some p in it:

$d_p=a_{f_p}{p}_x+b_{f_p}{p}_y+c_{f_p}---\left(4\right)$

P _xwith p _ythe coordinate of some p in triangle,

with triangle f _pthe parallax plane determining;

If triangle f _punreliable, to the pixel in it, adopt Bayesian model to ask parallax, with triangle f _pand the triangle that closes on common limit is as model priori, increases a parameterized level and smooth item as model priori simultaneously, Bayesian model is as follows:

P(D|I，F)∝P(I|D)P(F|D)P(D) (3)

Wherein D represents parallax, I representative image, and F represents triangle, P represents the probability in model;

According to Bayesian model, asking parallax is exactly the posterior probability problem of maximum value formula (3), and maximum value posterior probability is equivalent to the local energy function that minimal value is following:

E(p，f _p)=E _data(p，f _p)+λ _sE _smooth(p)+λ _fE _f(f _p) (5)

E (p, f _p) represent required triangle f _ptotal coupling cost of interior some p; E _data(p, f _p) be that postulated point p is at triangular facet f _pon coupling cost; E _smooth(p) it is level and smooth, for describing the relation between a p and consecutive point, E _f(f _p) be parallax face f _pinsecure penalty term, for describing f _pplane reliability, λ _sand λ _frespectively level and smooth and f _pthe control parameter of fiduciary level.

In step 3) during design of graphics, based on hypothesis: the pixel of the color similarity closing on should be positioned at same target, the two consistent pixels of depth information of closing on be positioned at same target, the weights on setting limit are determined jointly by color and the parallax of two pixels that are connected, if two pixels that in figure, limit is connected all comprise parallax information, by parallax and color according to the weights on the ratio-dependent limit of 7:3, otherwise, only according to colouring information, determine weights.

As optimal way, step 4), in, carry out for the first time after image cuts apart, carrying out post-processed: in conjunction with parallax information, the thinner cut zone that belongs to same parallax face of cutting apart of closing on is merged, for the fineness of cutting apart that judges whether to merge, according to image, cut apart Location of requirement here.

Step 5) in, need the judgment principle of the object that splits to be:

A) region of parallax maximum: the degree of depth is forward, object is close to camera lens, in the remarkable position of vision;

B) cut zone is compact: in cut zone, the ratio of number of pixels and cut zone place minimum rectangular area area is minimum;

C) ratio of the shared whole image-region of cut zone is moderate: Aspect Ratio meets predefined parameter:

$\frac{W\&times;H}{W_{\mathrm{all}}\&times;H_{\mathrm{all}}}>\frac{1}{250},\frac{W\&times;H}{W_{\mathrm{all}}\&times;H_{\mathrm{all}}}<\frac{1}{4},\frac{W}{H}<5$ And $\frac{H}{W}<5$

According to mentioned above principle, take out cut zone, obtain cutting object;

Wherein, establish q _numto belong to pixel sum in cut zone, d _avgthe mean parallax of pixel in cut zone, w _minthe minimum abscissa value of cut zone, w _maxmaximum abscissa value, h _minthe minimum ordinate value of cut zone, h _maxbe maximum ordinate value, cut zone is determined by formula (18) minimum rectangle, and W and H are respectively the length of this minimum rectangle and wide, W _allwith H _allthe length of entire image and wide:

W=w _max-w _min，H=h _max-h _min (18)

To the C that is in conformity with the principle) cut zone, according to each cut zone complexity O, sort from big to small, take out and to come n object above, the complexity of cutting apart is:

$O=d_{\mathrm{avg}}\&times;\frac{q_{\mathrm{num}}}{W\&times;H}---\left(19\right)$

Take out the minimum rectangle mask of cut zone, carry out binaryzation cavity and fill, then this mask is acted on to the object that former figure obtains splitting.

The present invention compared with prior art has the following advantages

The sectional perspective matching algorithm that the present invention proposes can effectively solve the existing sectional perspective matching algorithm based on trigonometric interpolation in the inaccurate problem of parallax discontinuity zone interpolation, and these regions comprise near object boundary and the Delta Region being comprised of abnormal initial matching point.Than the overall Stereo Matching Algorithm of present main flow, although the method disparity map lower and that obtain on matching precision is not dense, this does not affect the utilization of later stage on cut apart.The matching characteristic adopting due to the method is sobel Edge texture, so between object and object the abundant local parallax of textural characteristics mate more accurate, can effectively instruct Object Segmentation, and for the unconspicuous region of textural characteristics, although parallax information is inaccurate or disappearance, rely on colouring information also can successfully the pixel of same target be linked together.In addition, sharpest edges of sectional perspective matching process of the present invention are that time efficiency is high, have more practicality.

The Object Segmentation Algorithm based on color and the degree of depth that the present invention proposes can fast and effeciently be partitioned into a plurality of objects in image, compare with the global segmentation algorithm cutting based on figure of existing main flow, although segmentation effect is poor, but in cutting procedure, do not need man-machine interactively, cutting object be a plurality of and not background solely cut apart, and time efficiency is higher; Segmentation effect is compared with mean shift algorithm, and sliced time, efficiency was higher, and the non-over-segmentation of segmentation result.

Accompanying drawing explanation

Fig. 1 is implementing procedure of the present invention.

Fig. 2 is the implementing procedure of quick sectional perspective matching process of the present invention.

Fig. 3 is the implementing procedure of Object Segmentation of the present invention.

Fig. 4 is that the present invention carries out the model of interpolation to the point in unreliable triangle.

Embodiment

The present invention proposes a kind of stereo-picture Object Segmentation method based on quick local matching, comprise the following steps:

1) the left and right binocular stereo image after input validation, carries out routine sampling to left image, and the object of sampling is mainly that left and right image is all original input in order to guarantee the efficiency of algorithm, here left image is sampled, and right figure temporarily also has no relations.Then sampled point is used to the adaptive weighting sectional perspective matching process that classical matching precision is high, when the coupling cost of calculating pixel point, adopt the RGB colouring information of sobel textural characteristics and pixel, then matching result is used to later stage verification, as left and right consistency detection, obtain final match point reliably.

2) to step 1) the reliable matching point obtained uses Delaunay trigonometric ratio, and view picture image is covered by triangle gridding.To each triangle, utilize the triangle closing on to calculate his fiduciary level.If triangle is reliable, utilize parallax that this triangle determines to ask parallax in the face of the pixel interpolation in it; If unreliable, utilize this triangle and close on three altogether limit triangle pairs pixel interpolation in it ask parallax, the disparity map obtaining is carried out to later stage verification obtains final disparity map equally, this disparity map may be sparse.

3) to left picture construction figure, pixel is as the summit of figure, and pixel and its eight neighbour's line is as the limit of figure.The weights on limit are determined by color and the parallax of two pixels that are connected.

4) adopt Kruskal minimum spanning tree strategy, sorted from low to high according to the size of weights in limit, according to the weights on limit, determine whether the two pixel regions that are connected belong to same cut zone, and the threshold value of judgement changes according to region adaptivity.Handle successively all limits in figure, in picture, all pixels all incorporate corresponding region into.The easy over-segmentation of initial partitioning, in conjunction with parallax information, merges the thinner cut zone that belongs to same parallax face of cutting apart of closing in post-processed.

5) to the cut zone obtaining, according to parallax size, the compactness of cut zone and the scale of the shared picture in its entirety of cut zone, judge whether to belong to the object that need to obtain, take out cut zone, the object that obtains finally splitting.

If Fig. 1 is implementing procedure figure of the present invention, first to the left and right binocular stereo image after verification, utilize the method for existing adaptive weighting to ask reliable matching point.In order to guarantee the efficiency of algorithm, first left image is carried out to uniform sampling, the sample window that this method adopts is 3 * 3.For sampled point, by the method for classical adaptive weighting, in right image, search optimal match point.When assembling coupling cost according to local window in pixel self information and the definite weight of geometric distance carry out cost accumulative total and be added.Given pixel p, for the arbitrary pixel q in local window, the weight of q is by the color distortion c between p, q _pqwith Euclidean distance g _pqdetermine, in explanation of the present invention, the point in triangle is also pixel:

$\mathrm{\&omega;}(p,q)=\exp (-(\frac{c_{\mathrm{pq}}}{{\mathrm{\&sigma;}}_c}+\frac{g_{\mathrm{pq}}}{{\mathrm{\&sigma;}}_g}))---\left(6\right)$

σ _cwith σ _gthat the gathering cost C ' (p, d) of each pixel is as follows for controlling the parameter of color and Euclidean distance:

$C^{\&prime;}(p,d)={\mathrm{\&Sigma;}}_{q\&Element;{\mathrm{\&Omega;}}_p}\mathrm{\&omega;}(p,q)C(q,d)---\left(7\right)$

Ω _pthe local window of pixel p, C (q, d) be some q (x, y) in left image with right image in some q _dthe coupling cost of (x-d, y), the textural characteristics sobel difference that coupling cost of the present invention is pixel and the RGB colouring information of pixel.Then adopt the strategy of winner-take-all (winner-take-all) to find the parallax that mates Least-cost.

D(p)=arg min _d C′(p，d) (8)

D (p) is the parallax of pixel p, be the pixel p (x in left image, y) the pixel p (x-D (p) and in right image, y) coupling, all sampled points in left image are asked for after parallax, used post-processed, as left and right consistency check, remove some points not too reliably, final remaining point is exactly the reliable matching point needing.Use Delaunay trigonometric ratio that these reliable points are organized into triangle, all pixels of left image are all covered by triangle gridding, as shown in Figure 5.

To each triangle, utilize the triangle closing on to calculate his fiduciary level.In most cases, pixel in same target, their parallax should be positioned at same plane, thereby the triangle that is positioned at same target also should have similar parallax plane, if triangle is crossed over the border of object or formed leg-of-mutton point is insecure, the parallax face that this triangle determines is not suitable for that the point in it is carried out to interpolation and asks parallax, and it is insecure.Thereby the present invention proposes to utilize its triangle closing on to judge each leg-of-mutton reliability, the triangle that is applicable to reliably interpolation should be to its triangle around on similar parallax face.Judging whether triangle is positioned at same plane can be according to the angle of their planar process vector:

$\mathrm{\&theta;}={\cos }^{-1}\frac{\stackrel{\&RightArrow;}{n_{f_1}}\&CenterDot;\stackrel{\&RightArrow;}{n_{f_2}}}{\left|\right|\stackrel{\&RightArrow;}{n_{f_1}}\left|\right|\&CenterDot;\left|\right|\stackrel{\&RightArrow;}{n_{f_2}}\left|\right|}---\left(1\right)$

two triangle f ₁with f ₂normal vector, if angle theta is less than a certain threshold tau arranging in advance _f, two triangle f ₁with f ₂approximate think be positioned at same plane; Otherwise be positioned at Different Plane.If more these the leg-of-mutton reliabilities of coplanar triangle are also just higher, otherwise reliability is lower.The fiduciary level γ of triangle f (f) is calculated as:

$\mathrm{\&gamma;}\left(f\right)=\frac{t_r+{\mathrm{\&tau;}}_r}{t_n+{\mathrm{\&tau;}}_r},t_r\&le;t_n---\left(2\right)$

T _rthe adjacent triangle number coplanar with triangle f, t _nadjacent triangle total number, τ _rit is the parameter that fiduciary level is adjusted.

A) for reliable triangle γ (f _p)=1, its triangle closing on is all coplanar with it, and fiduciary level is 1, and the parallax plane of utilizing this triangle to determine is asked parallax to the point interpolation in it:

$d_p=a_{f_p}{p}_x+b_{f_p}{p}_y+c_{f_p}---\left(4\right)$

P _xwith p _ytriangle f _pthe coordinate of interior some p,

with

triangle f _pthe parallax plane determining;

B) for insecure triangle γ (f _p) < 1, utilize this triangle and be total to its interior pixel p interpolation of three triangle pairs of closing on limit and ask parallax.To the pixel in it, adopt Bayesian model to ask parallax, as Fig. 4, with triangle f _pand the triangle that closes on common limit is as model priori, increases a parameterized level and smooth item as model priori simultaneously, Bayesian model is as follows:

P(D|I，F)∝P(I|D)P(F|D)P(D) (3)

D is calculative parallax, and I is image itself, and F is the parallax face that triangle determines, is also triangle self, and P is the probability of Bayesian model.According to Bayesian model, asking parallax is exactly the problem of maximum value posterior probability, and maximum value posterior probability is equivalent to the local energy function that minimal value is following:

E(p，f _p)=E _data(p，f _p)+λ _sE _smooth(p)+λ _fE _f(f _p) (5)

E (p, f _p) be total coupling cost of a p; E _data(p, f _p) be that postulated point p is at triangle f _pcoupling cost on the parallax face determining; E _smooth(p) be level and smooth, as fruit dot p and adjacent pixel do not have corresponding penalty value, E at same parallax face _smooth(p) for describing the relation between a p and consecutive point, E _f(f _p) be parallax face f _pinsecure penalty term, for describing f _pplane reliability, λ _sand λ _frespectively level and smooth and f _pthe control parameter of fiduciary level.

In the present invention, the triangle of recording, parallax face and triangular facet three refer to same, and just, when mentioning this face from different perspectives, title is described upper some difference.For example, while describing from the angle of Delaunay trigonometric ratio, be called triangle, when the angle of parallax is described, be called parallax face, while describing from the angle of plane, be called triangular facet.

The disparity map obtaining is carried out to later stage verification obtains final disparity map equally, and this disparity map may be sparse.

Generally, suppose that in triangle, parallax is continually varying, for the point in triangle, adopt the definite parallax face of triangle to carry out interpolation and can determine parallax, as formula (4).Yet this simple interpolation is crossed over object boundaries to those or the triangle that is comprised of initial Exceptional point is inapplicable.According to most of real-life scenes, the parallax of putting in unreliable triangle is very likely on the reliable leg-of-mutton parallax face of its neighbour, so adopt the parallax face closing on effectively to carry out parallax interpolation to the point in unreliable triangle.The optimal number of closing on parallax face is that the detail by sampling density and image determines.In addition, time cost also needs to take into account, the computation complexity of method and the quantity correlation that closes on parallax face.Moreover, close on parallax face from reference point more away from, its impact on this point is less, so the present invention closes on 3 triangle f on common limit the most at last _n1, f _n2, f _n3as candidate's the parallax face that closes on, add the parallax face of intermediate triangle f, finally the candidate's parallax face for interpolation model forms by four:

${\mathrm{\&Theta;}}_{f_p}=\{f,f_{n1},f_{n2},f_{n3}\}---\left(9\right)$

In energy function (5), E _data(p, f _p) be that hypothesis pixel p is at parallax face f _pon coupling cost; E _smooth(p) be 4 neighbours level and smooth between matched pixel.Consider efficiency of algorithm, only adopt single treatment, the pixel that starts to have mated most may be fewer, but this can't cause relatively large deviation to result, because whole level and smooth is non-hard limit in energy function.

$E_{\mathrm{smooth}}\left(p\right)={\mathrm{\&Sigma;}}_{q\&Element;N_4}\mathrm{\&Delta;}{d}_{\mathrm{pq}}---\left(10\right)$

${\mathrm{\&Delta;d}}_{\mathrm{pq}}=\left\{\begin{array}{cc}1,& \mathrm{abs}(d_p-d_q)>1\\ 0,& \mathrm{otherwise}\end{array}\right.---\left(11\right)$

E _f(f _p) be plane f _punreliability penalty term, can determine f according to formula (2) _pfiduciary level.

$E_f\left(f_p\right)=\frac{1}{\mathrm{\&gamma;}\left(f_p\right)}---\left(12\right)$

Minimization energy function (5), finds best parallax face according to winner-take-all strategy

$f\left(p\right)={\arg \min }_{f_p\&Element;{\mathrm{\&Theta;}}_{f_p}}E(p,f_p)---\left(13\right)$

Find best parallax face, according to formula (4), calculate the parallax of a p.

The disparity map generating is used to post-processed, as left and right consistency check, obtain reliable sparse disparities figure.

Step 3) to left picture construction figure, as shown in Figure 3, pixel is as the summit of figure, and each pixel and its eight neighbour's line is as limit, and the weights on limit size determines jointly according to color and the depth information of two pixels that are connected:

If two pixels that are connected have reliable parallax value, color and parallax are according to the ratio-dependent weights of 3:7; Otherwise, only adopt colouring information as weights.

If p _r, p _g, p _bthe RGB triple channel color value of pixel p, q _r, p _g, p _bthe RGB triple channel color value of pixel p, color distortion W _cas formula (14)

$W_c=\sqrt{{(p_r-q_r)}^2+{(p_g-q_g)}^2+{(p_b-q_b)}^2}---\left(14\right)$

P _d, q _dpixel p, the parallax value of q, parallax difference W _das formula (15)

W _d=|p _d-q _d| (15)

The weight w on limit is as formula (16)

$W=\left\{\begin{array}{cc}0.7\&times;W_d+0.3\&times;W_c,& p_d\&NotEqual;0\mathrm{and}{q}_d\&NotEqual;0\\ 0.5\&times;W_c,& \mathrm{otherwise}\end{array}\right.---\left(16\right)$

P is two pixels that are connected with on one side with q, if p and q contain parallax, while determining weights, parallax difference accounts for 0.7, and color distortion accounts for 0.3; Otherwise get 0.5 of color distortion, determining weights, why get 0.5 here, is mainly in order to make color be unlikely to too strong to the impact of cutting apart in the situation that not considering parallax.

Built figure, adopted Kruskal ' s minimum spanning tree strategy to carry out image and cut apart, the complexity of cutting apart is O (mlogm), and concrete partitioning algorithm is as follows:

Input figure G=(V, E), output cut zone: S=(C ₁..., C _r)

Opposite side E sorts according to non-order of falling, and after sequence, is π=(o ₁..., o _m)

A. initial segmentation result is S ₀, each pixel is a cut zone

B. to every limit q=1 ..., m repeating step c

C. S on previous segmentation result basis _q-1, cut apart S next time _q, get q bar limit o _q=(v _i, v _jif). according to S _q-1segmentation result, v _iand v _jin different cut zone, and the weight w (o on limit _q) be less than the threshold value of two cut zone, by v _iand v _jtwo cut zone at place merge.Adjust the region threshold after merging.Otherwise S _q=S _q-1.

D. when handling all limits, return

In cutting procedure, the threshold value in region is adaptive change, is mainly in order to guarantee that segmentation result neither can over-segmentation can not cut apart very coarsely yet.

T(C)=Int(C)+τ(C)τ(C)=k／|C| (17)

T (C) is the threshold value of cut zone, and Int (C) is the maximum weights of connected pixel in cut zone, and τ (C) is the value of adaptive change.K is the threshold value arranging in advance, and it is generally used for the size of controlling cut zone, in whole cutting procedure, fixes.| C| is the size of cut zone, i.e. the number of pixel, and it is automatically to change.

Handle successively all limits in figure, in picture, all pixels all incorporate corresponding region into, obtain cut zone.

Although set region threshold, initial partitioning is still than being easier to over-segmentation, as optimal way, in post-processed in conjunction with parallax information, the thinner cut zone that belongs to same parallax face of cutting apart of closing on is merged, for the fineness of cutting apart that judges whether to merge, according to image, cut apart Location of requirement here.When initial partitioning, can cut apart more like this, then by post-processed, adjust, reach suitable segmentation result.

Step 5), in, to the region obtaining after cutting apart, need to take out wherein suitable object.Q _numto belong to pixel sum in cut zone; d _avgit is the mean parallax of pixel in cut zone; w _minthe minimum abscissa value of cut zone, w _maxit is maximum abscissa value; h _minthe minimum ordinate value of cut zone, h _maxit is maximum ordinate value.The cut zone of taking out can be determined by formula (18) minimum rectangle, and W and H are respectively the length of this minimum rectangle and wide, W _allwith H _allthe length of entire image and wide.

W=w _max-w _min，H=h _max-h _min (18)

Taking out the principle of object needing is

A. parallax is maximum: the degree of depth is the most forward, and object is nearest from camera lens, in the remarkable position of vision.

B. cut zone is compact: in cut zone, the ratio of number of pixels and cut zone place minimum rectangular area area is minimum;

C. the ratio of the shared whole image-region of cut zone is more moderate: Aspect Ratio is suitable, meets the threshold value of setting in advance.

The cut zone of a, b of being in conformity with the principle is directly taken out just passable, if the ratio of the shared entire image of cut zone is proper, the c that is in conformity with the principle, both not too little also not too greatly and Aspect Ratio suitable,

and

, according to the descending sequence of the size of each region O, take out and come n object above, the size of concrete n can freely be specified, and n object is exactly n cut zone.The complexity of cutting apart is:

$O=d_{\mathrm{avg}}\&times;\frac{q_{\mathrm{num}}}{W\&times;H}---\left(19\right)$

Take out the minimum rectangle mask of cut zone, carry out binaryzation cavity and fill, then this mask is acted on to the object that former figure obtains splitting.

The Stereo Matching Algorithm that the present invention proposes is effective fast, effectively the parallax discontinuity zone such as handled object edge; Partitioning algorithm can be partitioned into a plurality of objects rapidly, and time efficiency of the present invention is high, and segmentation effect is good, can meet the demand of fast automatic Object Segmentation.

Claims (7)

1. the stereo-picture Object Segmentation method based on quick local matching, is characterized in that comprising the following steps:

1) ask for reliable matching point: the left image after input validation, right image carries out uniform sampling to left image, to sampled point, use adaptive weighting sectional perspective matching process to search optimal match point in right image, when the coupling cost of calculating pixel point, adopt the RGB colouring information of sobel textural characteristics and pixel, then matching result is carried out to later stage verification, finally remaining point is as reliable matching point;

2) reliable matching point is used to Delaunay trigonometric ratio: the reliable matching point that left image is tried to achieve is organized into triangle gridding, each triangle is carried out to reliability judgement, judgement according to being to check that whether the triangle that triangle closes on it coplanar, point in diabolo, according to leg-of-mutton reliability, utilize three triangles on triangle and limit common with it to carry out interpolation and ask parallax; The disparity map obtaining is carried out to later stage verification and obtain final disparity map;

3) to left picture construction figure: each pixel of left image is as the summit of figure, the limit using pixel and its eight neighbour's line as figure, the weights on limit determine by color and the parallax information of the pixel that is connected, parallax information is obtained by final disparity map;

4) utilizing the figure building to carry out image cuts apart: adopt Kruskal minimum spanning tree strategy, sorted from low to high according to the size of weights in limit, according to the weights on limit, determine whether the two pixel regions that are connected belong to same cut zone, the threshold value of judgement changes according to region adaptivity, handle successively all limits in figure, in picture, all pixels all incorporate corresponding region into, obtain cut zone;

5) extract interested object in cut zone: to the cut zone obtaining, according to parallax size, the compactness of cut zone and the ratio of the shared picture in its entirety of cut zone, judge whether to belong to the object that need to split, according to object, divide the cut zone of taking out each object, complete Object Segmentation.

2. the stereo-picture Object Segmentation method based on quick local matching according to claim 1, is characterized in that described later stage verification adopts left and right consistency detection.

3. the stereo-picture Object Segmentation method based on quick local matching according to claim 1, is characterized in that described step 2) in, to each triangle, utilize its triangle closing on to carry out reliability judgement and be specially:

According to the angle theta of leg-of-mutton planar process vector, judge that whether triangle is coplanar:

$\mathrm{\&theta;}={\cos }^{-1}\frac{\stackrel{\&RightArrow;}{n_{f_1}}\&CenterDot;\stackrel{\&RightArrow;}{n_{f_2}}}{\left|\right|\stackrel{\&RightArrow;}{n_{f_1}}\left|\right|\&CenterDot;\left|\right|\stackrel{\&RightArrow;}{n_{f_2}}\left|\right|}---\left(1\right)$

two triangle f ₁with f ₂normal vector, if angle theta is less than the threshold tau setting in advance _f, two triangle f ₁with f ₂think and be positioned at same plane, coplanar; Otherwise be positioned at Different Plane;

The fiduciary level γ of triangle f (f) is calculated as:

$\mathrm{\&gamma;}\left(f\right)=\frac{t_r+{\mathrm{\&tau;}}_r}{t_n+{\mathrm{\&tau;}}_r},t_r\&le;t_n---\left(2\right)$

T _rthe adjacent triangle number coplanar with triangle f, t _ntriangle total number adjacent with triangle f, τ _rit is the parameter that fiduciary level is adjusted.

4. the stereo-picture Object Segmentation method based on quick local matching according to claim 3, is characterized in that step 2) in, according to leg-of-mutton reliability, ask parallax to be specially:

If triangle f _preliably, its triangle closing on is all coplanar with it, and fiduciary level is 1, utilizes parallax that triangle determines to ask parallax in the face of the some p in it:

$d_p=a_{f_p}{p}_x+b_{f_p}{p}_y+c_{f_p}---\left(4\right)$

P _xwith p _ythe coordinate of some p in triangle,

with

triangle f _pthe parallax plane determining;

If triangle f _punreliable, to the pixel in it, adopt Bayesian model to ask parallax, with triangle f _pand the triangle that closes on common limit is as model priori, increases a parameterized level and smooth item as model priori simultaneously, Bayesian model is as follows:

P(D|I，F)∝P(I|D)P(F|D)P(D) (3)

Wherein D represents parallax, I representative image, and F represents triangle, P represents the probability in model;

According to Bayesian model, asking parallax is exactly the posterior probability problem of maximum value formula (3), and maximum value posterior probability is equivalent to the local energy function that minimal value is following:

E(p，f _p)=E _data(p，f _p)+λ _sE _smooth(p)+λ _fE _f(f _p) (5)

E (p, f _p) represent required triangle f _ptotal coupling cost of interior some p; E _data(p, f _p) be that postulated point p is at triangular facet f _pon coupling cost; E _smooth(p) it is level and smooth, for describing the relation between a p and consecutive point, E _f(f _p) be parallax face f _pinsecure penalty term, for describing f _pplane reliability, λ _sand λ _frespectively level and smooth and f _pthe control parameter of fiduciary level.

5. the stereo-picture Object Segmentation method based on quick local matching according to claim 1, it is characterized in that in step 3) during design of graphics, based on hypothesis: the pixel of the color similarity closing on should be positioned at same target, the two consistent pixels of depth information of closing on be positioned at same target, the weights on setting limit are determined jointly by color and the parallax of two pixels that are connected, if two pixels that in figure, limit is connected all comprise parallax information, by parallax and color according to the weights on the ratio-dependent limit of 7:3, otherwise, only according to colouring information, determine weights.

6. the stereo-picture Object Segmentation method based on quick local matching according to claim 1, it is characterized in that step 4) in, carry out for the first time after image cuts apart, carry out post-processed: in conjunction with parallax information, the thinner cut zone that belongs to same parallax face of cutting apart of closing on is merged, for the fineness of cutting apart that judges whether to merge, according to image, cut apart Location of requirement here.

7. the stereo-picture Object Segmentation method based on quick local matching according to claim 1, is characterized in that step 5) in need the judgment principle of the object that splits to be:

A) region of parallax maximum: the degree of depth is forward, object is close to camera lens, in the remarkable position of vision;

B) cut zone is compact: in cut zone, the ratio of number of pixels and cut zone place minimum rectangular area area is minimum;

C) ratio of the shared whole image-region of cut zone is moderate: Aspect Ratio meets predefined parameter:

$\frac{W\&times;H}{W_{\mathrm{all}}\&times;H_{\mathrm{all}}}>\frac{1}{250},\frac{W\&times;H}{W_{\mathrm{all}}\&times;H_{\mathrm{all}}}<\frac{1}{4},\frac{W}{H}<5$ And $\frac{H}{W}<5$

According to mentioned above principle, take out cut zone, obtain cutting object;

Wherein, establish q _numto belong to pixel sum in cut zone, d _avgthe mean parallax of pixel in cut zone, w _minthe minimum abscissa value of cut zone, w _maxmaximum abscissa value, h _minthe minimum ordinate value of cut zone, h _maxbe maximum ordinate value, cut zone is determined by formula (18) minimum rectangle, and W and H are respectively the length of this minimum rectangle and wide, W _allwith H _allthe length of entire image and wide:

W=w _max-w _min，H=h _max-h _min (18)

To with the C that is in conformity with the principle) cut zone, according to each cut zone complexity O, sort from big to small, take out and to come n object above, the complexity of cutting apart is:

$O=d_{\mathrm{avg}}\&times;\frac{q_{\mathrm{num}}}{W\&times;H}---\left(19\right)$

Take out the minimum rectangle mask of cut zone, carry out binaryzation cavity and fill, then this mask is acted on to the object that former figure obtains splitting.

Images