CN102611904A - Stereo matching method based on image partitioning in three-dimensional television system - Google Patents

Abstract

The invention discloses a stereo matching method based on an image partitioning algorithm in a faster self-adaptive three-dimensional television system, aiming at obtaining partitioning information of a reference image and applying the partitioning information to the three-dimensional television system, wherein the method is better in instantaneity, accurate in obtaining depth information and strong in anti-interference ability for image noise (lighting, distortion and the like). The method comprises the following specific steps of: firstly, shooting left and right images using a parallel binocular camera and calibrating the images; secondly, calculating an initial parallax error by using an improved cross polymerization algorithm so as to obtain an initial parallax error image of a left image or a right image; thirdly, adopting the image partitioning algorithm based on a graph theory to partition the left image or the right image calibrated in the first step using the image partitioning algorithm based on the graph theory; and fourthly, carrying out median filtering to the initial parallax error image information obtained in the second step by using partitioned images so as to obtain a parallax error image of the left image or the right image needing to be matched.

Description

In the three-dimensional television system based on the solid matching method of image segmentation

Technical field

The present invention relates to a kind of treatment of picture method, relate in particular in a kind of three-dimensional television system solid matching method based on image segmentation.

Background technology

Three-dimensional coupling is the core part of computer vision field, in practices such as three-dimensional television, automobile assistant driving, computer vision, all is widely used.Difference according to the coupling primitive; Be broadly divided into based on the matching algorithm of characteristic with based on the matching algorithm of area; Though and based on the solid matching method rapid speed of characteristic but owing to can not obtain global optimum's parallax; So global effect is better used more extensively based on the matching algorithm of area in recent years, wherein figure cuts precision and the accuracy that algorithm (graph-cuts), Dynamic Programming (Dynamic Programming) algorithm and belief propagation (Belief propagation) algorithm stereo mate and improves a lot.

Jiangbo Lu, Ke Zhang, Lafruit; G.; Catthoor, F. be at acoustics, voice and signal processing meeting (Acoustics; Speech and Signal Processing; 2009.) on the paper " a kind of real-time solid matching method " (Real-time stereo matching:A cross-based local approach) delivered based on the cross polymerization a kind of cross polymerization Stereo Matching Algorithm has been proposed, this algorithm is a kind of matching algorithm of sectional perspective more fast, amount of calculation is less and effect is pretty good.But the result that this algorithm obtains global registration algorithm bigger than amount of calculation on accuracy is poor.

Tao H; Sawhney H S and Kumar R is published in the 8th computer vision meeting of Vancouver, CAN (Proceedings of the Eighth International Conference On Computer Vision.Vancouver in calendar year 2001; Canada; 2001 (1): the paper 532-539) " a kind of Stereo Matching Algorithm framework based on color images " (A Global Matching Framework for Stereo Computation) can use level and smooth parallax model representation and the discontinuous place of parallax with the consistent characteristic in cut zone edge according to the inner parallax of color smooth region; Earlier coloured image is cut apart; Utilize rapid speed then but the not high matching algorithm of accuracy rate to left and right sides image calculation initial parallax figure, utilize carve information that the initial parallax of each cut zone is carried out match at last.The advantage of this algorithm is that the parallax border ordinary circumstance that obtains of single image is more more accurate than the border that is obtained by disparity estimation, and the robustness of occlusion area coupling is also improved, and its efficiency of algorithm is higher relatively.Its shortcoming be since partitioning algorithm influence its cut apart hypothesis might not be always correct, and owing to force level and smooth influence, the parallax that obtains possibly be able to not be represented the parallax that the zone is real.

Mean shift (mean-shift) partitioning algorithm is a kind of method of the carve information that obtains image commonly used; The mean shift algorithm is a kind of non-parametric estmation method of utilizing the gradient searching peak value of probability distribution; Has higher robustness; It is cut apart image through the color gamut of input color image and space domain characteristic are carried out cluster, and this algorithm can obtain segmentation effect preferably, and the shortcoming of this algorithm is that real-time is general.

Summary of the invention

For remedying the deficiency of prior art, the present invention propose a kind of speed faster, in the adaptive three-dimensional television system based on the solid matching method of image segmentation.At first utilize improved cross polymerization Stereo Matching Algorithm to calculate initial parallax, the carve information of using the image segmentation based on graph theory to obtain then is optimized initial parallax.Be applied in the three-dimensional television system, computational speed is faster, and real-time is better, and it is accurate to obtain depth information, and strong for the antijamming capability of picture noise (illumination, distortion etc.).This method is applicable to that the degree of depth in the three-dimensional television system generates application, also can be widely used in practices such as automobile assistant driving, computer vision.

For realizing above-mentioned purpose, the present invention adopts following technical scheme:

Based on the solid matching method of image segmentation, concrete steps are following in a kind of three-dimensional television system:

The first step is calibrated with parallel binocular camera shooting left and right sides image and to it, makes image in two width of cloth images, be in same horizontal line for the same pixel of same object, to meet the restrictive condition of three-dimensional coupling;

Second step; Utilize improved cross polymerization Stereo Matching Algorithm to calculate initial parallax; Left and right sides image to after the calibration is analyzed; Obtain each pixel corresponding cross zone; Further obtaining the corresponding adaptive region of each pixel, calculate the coupling cost based on adaptive region then, is optimal solution with the minimum parallax of coupling cost at last; Obtain the initial parallax of left figure or right figure, and with the initial parallax of this initial parallax as the 4th step;

The 3rd step; Employing is based on the image segmentation algorithm of graph theory; Use the image segmentation algorithm based on graph theory to cut apart to left figure after the calibration in the first step or right desiring to make money or profit, the return value S that obtains is the final carve information of figure, and it belongs to different cut zone with the summit with a series of label information sign; Corresponding relation by pixel and summit; Thereby just obtained the carve information of image, the intensity parameters c of a colour has been set, the cut zone of number of pixels in the segmentation result less than c merged in the big block on every side;

The 4th step, utilize the image after cutting apart that the initial parallax information that obtains in second step is carried out medium filtering, needing to obtain the left figure of coupling or the disparity map of right figure.

In the said first step, because the interference of the difference of camera parameter and lens distortion needs earlier image to be calibrated, make image in two width of cloth images, be in same horizontal line, to meet the restrictive condition of three-dimensional coupling for the same pixel of same object; Selection standard image Tsukuba image carries out the calculating of initial parallax to as original image for this reason.

In said second step; Left and right sides image after adopting cross polymerization algorithm to calibration is analyzed; Obtain each pixel corresponding cross zone, further obtain the corresponding adaptive region of each pixel, calculate the coupling cost based on adaptive region then; Be optimal solution with the minimum parallax of coupling cost at last, obtain the initial parallax of left figure or right figure.Its concrete steps are:

2-1) to two width of cloth graphical analyses of needs coupling, obtain each pixel corresponding cross zone

To the pixel p=(x on the input picture _p, y _p), x _p, y _pRepresent the horizontal ordinate of p point position respectively, with { h _p ^-, h ⁺ _p, v ^- _p, v ⁺ _pFour parameters represent to change the left side in pixel cross zone, right side, upside, downside brachium respectively.Utilize on the cross zone color similarity degree of pixel and central point to confirm each brachium.For example to confirm left arm length h _p ^-, detect left pixel point p left successively from central point ₁With the similarity of central point p, judge whether that similar function does

$\mathrm{\δ}(p_1,p)=\left\{\begin{array}{cc}1,& \underset{c\∈\{R,G,B\}}{\max }\left(\right|I_c\left(p_1\right)-I_c\left(p\right)\left|\right)\≤\mathrm{\τ}\\ 0,& \mathrm{otherwise}\end{array}\right.$

I _cBe colored intensity (c ∈ { R, G, B}, I of c _cPossible value I _R, I _G, I _BRepresent the RGB colouring intensity respectively).Parameter τ is a color similarity degree threshold value, and τ=10 are set here.

As δ (p ₁, p)=1 o'clock, represent similar.Up to detecting δ (p ₁, p)=0, find first dissimilar point of left side, confirmed left arm length.To each brachium, limiting maximum brachium is 5, and minimum brachium is 1.If promptly detection arm is grown up in 5 and is set to 5, brachium is set to 1 less than 1.In like manner can confirm the right side, upside, downside brachium.Each pixel corresponding cross zone of two width of cloth images about this step can obtain.

2-2) obtain the adaptive region of each pixel

For each pixel; Search for each point of the vertical direction (being its upper arm, underarm) in its cross zone; The horizontal zone (be its left arm, right arm) corresponding these points also adds the zone of convergency: whole like this zone is exactly the adaptive region of an indefinite shape, representes this adaptive region with U (p).

2-3) calculate the coupling cost based on adaptive region

At first calculate the coupling cost of each pixel $e_d\left(s\right)=\underset{c\∈\{R,G,B\}}{\mathrm{\Σ}}|I_c\left(s\right)-{I^{\′}}_c\left(s^{\′}\right)|,$

D wherein is a parallax value, and s is the point on the left figure, and s ' is the point on the right figure.I _cBe colored (c ∈ { R, G, intensity B}) of c.

Calculate the normalization coupling cost on this coincidence zone $\stackrel{\‾}{E}\left(p\right)=\frac{1}{\left|\right|U_d\left(p\right)\left|\right|}{E}_d\left(p\right)=\frac{1}{\left|\right|U_d\left(p\right)\left|\right|}\underset{s\∈U_d\left(p\right)}{\mathrm{\Σ}}{e}_d\left(s\right).$ U wherein _d(p)=(x, y) | (x, y) ∈ U (p), (x-d, y) ∈ U ' (p ') }, represent adaptive region U (p) and right figure point p ' on the left figure point p adaptive region U ' (p ') overlap the zone.(x in the formula, y are the horizontal ordinate of pixel in image, and d is a parallax).

|| U _d(p) || be regional U _d(p) the pixel number that comprises.

In like manner can obtain with right side figure is the matching result

of benchmark

2-4) choose the minimum parallax of normalization coupling cost optimal solution the most, obtain initial parallax

Each pixel p of left figure so that minimum parallax d is an optimal solution, is obtained initial parallax; In like manner; If the pixel p ' among the right figure obtains initial parallax so that

minimum parallax d is an optimal solution.

In said the 3rd step, following based on the image segmentation algorithm step of graph theory:

3-1) limit collection E is arrived π=(o by the sequence arrangement that weights increase progressively ₁..., o _m);

3-2) make that initial segmentation is S ⁰, each summit among the V all is initial segmentation S ⁰In isolated component;

3-3) for q=1 ... .m, repeating step 3-4);

3-4) by S ^Q-1Make up S ^qThe time, make the q bar limit o among the π _qThe node that connects is v _i, v _j, i.e. o _q=(v _i, v _j), if v _i, v _jBelong to S respectively ^Q-1In different components, and w (o _q) all littler than two component internal diversities, then merge two components, otherwise, nonjoinder; Even Be S ^Q-1In comprise v _iComponent,

Be to comprise v _jComponent, if satisfy

And w (o _q) less than

With

Threshold value T in any one, then S ^Q-1In With

Be merged into one-component, obtain S ^q, otherwise S ^q=S ^Q-1, i.e. nonjoinder;

3-5) return S=S ^mThe S that returns is the carve information of figure, and it belongs to different cut zone with the summit with a series of label information sign, by the corresponding relation on pixel and summit, can obtain the carve information of image.

Beneficial effect of the present invention:

1. utilization of the present invention obtains disparity map based on the image segmentation algorithm of graph theory, and this method not only can obtain disparity map more accurately, and the Stereo Matching Algorithm of cutting apart based on mean-shift relatively, and its speed of service also is greatly improved.

2. when calculating initial parallax figure, we adopt improved cross polymerization algorithm, and this algorithm computation result is accurate and amount of calculation is little.Improvement to cross polymerization algorithm is mainly reflected in the simplification of two aspects:

2.1, use improving in the 3rd step of cross polymerization algorithm

Replace $e_d\left(s\right)=\mathrm{Min}\left(\underset{c\∈\{R,G,B\}}{\mathrm{\Σ}}\right|I_c\left(s\right)-{I^{\′}}_c\left(s^{\′}\right)|,T)\×\frac{255}{T},$ (parameter T ∈ [0,765] is interceptive value, generally rule of thumb chooses certain bigger value.) be that the coupling cost is not blocked qualification in this method.

2.2 in the 4th step of improved cross polymerization algorithm, directly choose the minimum parallax of normalization coupling cost optimal solution the most, replace utilizing in the original text greedy algorithm (Winner-Take-All) the coupling cost to be handled the method for more insincere zone being filled.

Simplify at this two place, has reduced the amount of calculation of cross polymerization algorithm, and initial parallax has been obtained in employing method more fast and effectively, has improved computational speed.

3. be applied in the three-dimensional television system, can effectively overcome interference of noise such as illumination, obtained good effect.In the three-dimensional television system, require used Stereo Matching Algorithm can obtain more accurate matching result, and algorithm complex can not be too big, speed can not be too slow.This method be a kind of matching result accurately and the algorithm of rapid speed.Be adapted to very much the requirement of three-dimensional television system.For stereoscopic vision, application such as automobile assistant driving, this method also can obtain bigger application.

Description of drawings

Fig. 1 is a FB(flow block) of the present invention;

Fig. 2 a is the left figure of standard picture Tsukuba;

Fig. 2 b is the right figure of standard picture Tsukuba;

Fig. 2 c is the disparity map that obtains based on the Mean-shift partitioning algorithm, and be 9.13s running time;

Fig. 2 d is the segmentation result figure based on graph theory, and parameter is set to k=450, c=24;

Fig. 2 e is the disparity map that utilizes partitioning algorithm of the present invention to obtain, and parameter is set to k=450, c=25, and be 6.24s running time;

Fig. 2 f is the disparity map that utilizes partitioning algorithm of the present invention to obtain, and parameter is set to k=550, c=40, and be 6.31s running time;

Fig. 2 g is the disparity map that utilizes partitioning algorithm of the present invention to obtain, and parameter is set to k=650, c=25, and be 6.06s running time;

Fig. 3 a is the left figure of the pavement image of shooting;

Fig. 3 b is the right figure of the pavement image of shooting;

Fig. 3 c is the disparity map that utilizes Fig. 3 a that belief propagation algorithm obtains;

Fig. 3 d is the disparity map that utilizes Fig. 3 a that algorithm of the present invention obtains.

Embodiment

Below in conjunction with accompanying drawing and embodiment the present invention is described further:

As shown in Figure 1, based on the solid matching method of image segmentation, the performing step of this method is following in the three-dimensional television system:

1) obtains the left and right sides image of the target that needs coupling and it is calibrated

In practical application, we need at first with parallel binocular camera shooting left and right sides image, are depicted as the left and right sides image of pavement image like Fig. 3 a and Fig. 3 b; Because the interference of the difference of camera parameter and lens distortion needs earlier image to be calibrated, make image in two width of cloth images, be in same horizontal line, to meet the restrictive condition of three-dimensional coupling for the same pixel of same object.In the present invention, for the ease of the performance of this algorithm better is described, selection standard image Tsukuba image carries out the calculating of initial parallax to as original image.

2) utilize improved cross polymerization algorithm computation initial parallax

This method mainly comprises following steps:

1. to two width of cloth graphical analyses of needs coupling, obtain each pixel corresponding cross zone

To the pixel p=(x on the input picture _p, y _p), x _p, y _pRepresent the horizontal ordinate of p point position respectively, with { h _p ^-, h ⁺ _p, v ^- _p, v ⁺ _pFour parameters represent to change the left side in pixel cross zone, right side, upside, downside brachium respectively.Utilize on the cross zone color similarity degree of pixel and central point to confirm each brachium.For example to confirm left arm length h _p ^-, detect left pixel point p left successively from central point ₁With the similarity of central point p, judge whether that similar function does

$\mathrm{\δ}(p_1,p)=\left\{\begin{array}{cc}1,& \underset{c\∈\{R,G,B\}}{\max }\left(\right|I_c\left(p_1\right)-I_c\left(p\right)\left|\right)\≤\mathrm{\τ}\\ 0,& \mathrm{otherwise}\end{array}\right.$

I _cBe colored intensity (c ∈ { R, G, B}, I of c _cPossible value I _R, I _G, I _BRepresent the RGB colouring intensity respectively).Parameter τ is a color similarity degree threshold value, and τ=10 are set here.

As δ (p ₁, p)=1 o'clock, represent similar.Up to detecting δ (p ₁, p)=0, find first dissimilar point of left side, confirmed left arm length.To each brachium, limiting maximum brachium is 5, and minimum brachium is 1.If promptly detection arm is grown up in 5 and is set to 5, brachium is set to 1 less than 1.In like manner can confirm the right side, upside, downside brachium.Each pixel corresponding cross zone of two width of cloth images about this step can obtain.

2. obtain the adaptive region of each pixel

For each pixel; Search for each point of the vertical direction (being its upper arm, underarm) in its cross zone; The horizontal zone (be its left arm, right arm) corresponding these points also adds the zone of convergency: whole like this zone is exactly the adaptive region of an indefinite shape, representes this adaptive region with U (p).

3. calculate the coupling cost based on adaptive region

At first calculate the coupling cost of each pixel $e_d\left(s\right)=\underset{c\∈\{R,G,B\}}{\mathrm{\Σ}}|I_c\left(s\right)-{I^{\′}}_c\left(s^{\′}\right)|,$ D wherein is a parallax value, and s is the point on the left figure, and s ' is the point on the right figure.I _cBe colored (c ∈ { R, G, intensity B}) of c.

Calculate the normalization coupling cost on this coincidence zone $\stackrel{\‾}{E}\left(p\right)=\frac{1}{\left|\right|U_d\left(p\right)\left|\right|}{E}_d\left(p\right)=\frac{1}{\left|\right|U_d\left(p\right)\left|\right|}\underset{s\∈U_d\left(p\right)}{\mathrm{\Σ}}{e}_d\left(s\right).$ U wherein _d(p)=(x, y) | (x, y) ∈ U (p), (x-d, y) ∈ U ' (p ') }, represent adaptive region U (p) and right figure point p ' on the left figure point p adaptive region U ' (p ') overlap the zone.(x in the formula, y are the horizontal ordinate of pixel in image, and d is a parallax).

|| U _d(p) || be regional U _d(p) the pixel number that comprises.

In like manner can obtain with right side figure is the matching result

of benchmark

4. choose the minimum parallax of normalization coupling cost optimal solution the most, obtain initial parallax

Each pixel p of left figure so that

minimum parallax d is an optimal solution, is obtained initial parallax; In like manner; If the pixel p ' among the right figure obtains initial parallax so that

minimum parallax d is an optimal solution.

3) based on the image segmentation algorithm of graph theory

Based on the image segmentation algorithm of graph theory at first make up a cum rights non-directed graph G=(V, E), the pixel in the set V correspondence image on summit, the difference value of neighbor is mapped as the limit collection E of the undirected value of cum rights among the figure, connects neighbor pixel x _i, x _jLimit o (x _k, x _l) weight w (x _i, x _j) diversity factor between having reflected at 2, generally we elect the poor of gray value as.The simple method of carrying out image segmentation is with weight w (x _i, x _jAll remove on the limit of)＞t (t is the threshold value of weights), searches the remaining block that links to each other then.As long as but, will cause two zones that differ bigger to be merged because this kind algorithm has monolateral linking to each other.Therefore not enough with the method robustness of this connected region, also be difficult to find a suitable t value.Utilizing method and a threshold value T of minimum spanning tree is a kind of effective way that is communicated with the block cluster.Crewe Si Keer (Kruskal) algorithm is a kind of greedy algorithm, can provide a kind of minimum spanning tree (MST) of optimum; Begin a complete connectionless figure, according to the weights on limit, under the condition that does not form ring, one one increase limit is with threshold value w (x _i, x _jThe limit of)＞t removes, and obtains the final connection block that needs.

(the Efficient graph-based image segmentation.Int.J.of Comp.Vis. of the image segmentation algorithm based on graph theory that P.F.Felzenszwalb and D.P.Huttenlocher delivers on the computer vision periodical of 2004 the 2nd the 59th phases of volume; 59 (2): 167-181; 2004) provide improving one's methods of Crewe Si Keer algorithm, utilized the method for adaptive threshold to make algorithm more effective.The same with Crewe Si Keer algorithm, also be a complete connectionless figure at the beginning, increase the limit according to weights, current block is built up a minimum spanning tree.And each minimum spanning tree C _iAn internal diversity degree is all arranged, and also is threshold value

T(C _i)＝w(C _i)+k/|C _i| (4)，

W (C wherein _i) be maximum in a minimum spanning tree value, k is a constant, | C _i| be the number of pixel in the block, we can decide the size of cut zone through the value of parameter k is set.Suppose that we will handle limit (x _k, x _l), its two end points are two independently MST x _iAnd x _jIf, w (x at this moment _k, x _l)≤min (T (C _i), T (C _j)), then two MST just can merge.

Based on above theory, the partitioning algorithm step that can get the described graph theory among the present invention is following:

Suppose input Fig. 2 a, (V, E), this non-directed graph has n summit and m bar limit to its cum rights non-directed graph G=, is output as a kind of S=of cutting apart (C of V ₁..., C _r), the summit that is about to same cut zone gives consistent label, and sign belongs to the same area, and different cut zone labels is different, C ₁, C ₂..., C _rI.e. expression has different cut zone.

1. the difference value of all neighbors among Fig. 2 a is mapped as the limit collection E of the undirected value of cum rights among the figure, the sequence arrangement that limit collection E is increased progressively by weights arrives set π=(o ₁..., o _m);

2. make that initial segmentation is S ⁰(initial segmentation promptly corresponds to original image), each summit among the V (each pixel among each summit correspondence image 2a) all is initial segmentation S ⁰In isolated component;

3. for q=1 ... .m, repeating step 4; (S ^qSegmentation result when being the q time circulation completion);

4. by S ^Q-1Make up S ^qThe time, make the q bar limit o among the π _qThe node that connects is v _i, v _j, i.e. o _q=(v _i, v _j), if v _i, v _jBelong to S respectively ^Q-1In different components, and the weight w (o on limit _q) all littler than two component internal diversities (being threshold value T (C)), then merge two components, otherwise, nonjoinder.That is,

Be S ^Q-1In comprise v _iMinimum spanning tree,

Be to comprise v _jMinimum spanning tree, if satisfy

And w (o _q) less than

With Any of threshold value T (C), then S ^Q-1In

With

Be merged into one-component, obtain S ^q, otherwise S ^q=S ^Q-1, i.e. nonjoinder;

5. return S=S ^m(S ^mFor scheming cutting apart after m circulation accomplished)

Through above graph theory partitioning algorithm; The return value S that obtains is the final carve information of figure; It belongs to different cut zone with the summit with a series of label information sign, and by the corresponding relation on pixel and summit, we have also just obtained the carve information of image; We are provided with a parameter c, and the cut zone of number of pixels in the segmentation result less than c merged in the big block on every side.Shown in Fig. 2 d, be the segmentation result figure of Fig. 2 a based on graph theory, parameter is set to k=450, c=25.

4) utilize carve information to optimize parallax

A last step utilizes image segmentation algorithm to obtain the final carve information of image.Be about to image segmentation and become a lot of zones.We think each the zone in parallax be identical, in each zone, utilize the method for medium filtering that initial parallax is carried out filtering, with filtered parallax value as this regional parallax, shown in Fig. 2 e, f, g.

Experimental situation of the present invention is: AMD athlon (tm) 64X2Dual core Processor 4800+2.51GHz, and the 2.00GB internal memory uses matlab emulation; Obtain a series of experimental result pictures of Fig. 2 a-g and Fig. 3 a-d, wherein: Fig. 2 a, b are the standard picture that vision.middlebury.edu/stereo downloads from the website, among Fig. 2 e; Parameter is set to k=450, c=25, and be 6.24s running time; Wherein parameter k is for the threshold value of control cut zone size, and along with the increase of k, the zone of cutting apart increases; C can merge elimination for the parameter of control Minimum Area area to the very few cut zone of pixel quantity; Among Fig. 2 f, parameter is set to k=550, c=40, and be 6.31s running time, compares with Fig. 2 e, along with the increase of c, more little zone is merged in the image; Among Fig. 2 g, parameter is set to k=650, c=25, and be 6.06s running time, compares with Fig. 2 e, along with the increase of k, cut zone increases; Can know that by Fig. 2 e, f, g method of the present invention can obtain disparity map more accurately, and on speed; About 45% raising has been arranged with respect to the mean-shift algorithm; Shown in Fig. 2 c, its arithmetic result figure for cutting apart based on Mean-shift, be 9.13s running time.

For algorithm superiority of the present invention better is described; We have taken Fig. 3 a, b, and for the bigger pavement image of picture noise, the present invention is with respect to belief propagation algorithm (BP); Can obtain more accurate disparity map, be depicted as the disparity map that utilizes Fig. 3 a that belief propagation algorithm obtains like Fig. 3 c; Fig. 3 d is the disparity map that utilizes Fig. 3 a that algorithm of the present invention obtains, and compares with Fig. 3 c, and the parallax information of image is more accurate, and is stronger for the antijamming capability of illumination noise.

Claims (3)

1. In the three-dimensional television system based on the solid matching method of image segmentation, it is characterized in that concrete steps are following:

   The first step is taken left and right sides image and to its calibration, is made image in two width of cloth images, be in same horizontal line for the same pixel of same object with parallel binocular camera, to meet the restrictive condition of three-dimensional coupling;

   Second step; Utilize improved cross polymerization Stereo Matching Algorithm to calculate initial parallax; Promptly the left and right sides image after the calibration is analyzed; Obtain each pixel corresponding cross zone; Further obtaining the corresponding adaptive region of each pixel, calculate the coupling cost based on adaptive region then, is optimal solution with the minimum parallax of coupling cost at last; Obtain the initial parallax of left figure or right figure, and with the initial parallax of this initial parallax as the 4th step;

   The 3rd step; Employing is based on the image segmentation algorithm of graph theory; Use the image segmentation algorithm based on graph theory to cut apart to left figure after the calibration in the first step or right desiring to make money or profit, the return value S that obtains is the final carve information of figure, and it belongs to different cut zone with the summit with a series of label information sign; Corresponding relation by pixel and summit; Thereby just obtained the carve information of image, a parameter c has been set, the cut zone of number of pixels in the segmentation result less than c merged in the big block on every side;

   The 4th step, utilize the image after cutting apart that the initial parallax information that obtains in second step is carried out medium filtering, needing to obtain the left figure of coupling or the disparity map of right figure.
2. 2. based on the solid matching method of the image segmentation algorithm of graph theory, it is characterized in that in the three-dimensional television system as claimed in claim 1 that in said second step, its concrete steps are:

   2-1) to two width of cloth graphical analyses of needs coupling, obtain each pixel corresponding cross zone

   To the pixel p=(x on the input picture _p, y _p), x _p, y _pRepresent the horizontal ordinate of p point position respectively, with { h _p ^-, h ⁺ _p, v ^- _p, v ⁺ _pFour parameters represent to change the left side in pixel cross zone, right side, upside, downside brachium respectively; Utilize on the cross zone color similarity degree of pixel and central point to confirm each brachium; Confirm left arm length h _p ^-The time, detect left pixel point p left successively from central point ₁With the similarity of central point p, judge whether that similar function does

   $\mathrm{\δ}(p_1,p)=\left\{\begin{array}{cc}1,& \underset{c\∈\{R,G,B\}}{\max }\left(\right|I_c\left(p_1\right)-I_c\left(p\right)\left|\right)\≤\mathrm{\τ}\\ 0,& \mathrm{otherwise}\end{array}\right.$

   I _cBe the colored intensity of c, c ∈ { R, G, B}, I _cPossible value I _R, I _G, I _BRepresent the RGB colouring intensity respectively; Parameter τ is a color similarity degree threshold value, and τ=10 are set here;

   As δ (p ₁, p)=1 o'clock, represent similar; Up to detecting δ (p ₁, p)=0, find first dissimilar point of left side, confirmed left arm length; To each brachium, limiting maximum brachium is 5, and minimum brachium is 1, if promptly detection arm is grown up in 5 and is set to 5, brachium is set to 1 less than 1; In like manner can confirm the right side, upside, downside brachium; Each pixel corresponding cross zone of two width of cloth images about this step can obtain;

   2-2) obtain the adaptive region of each pixel

   For each pixel; The vertical direction of searching for its cross zone is each point of its upper arm, underarm; The horizontal zone corresponding these points is that its left arm, right arm also add the zone of convergency: whole like this zone is exactly the adaptive region of an indefinite shape, representes this adaptive region with U (p);

   2-3) calculate the coupling cost based on adaptive region

   At first calculate the coupling cost of each pixel $e_d\left(s\right)=\underset{c\∈\{R,G,B\}}{\mathrm{\Σ}}|I_c\left(s\right)-{I^{\′}}_c\left(s^{\′}\right)|,$ D wherein is a parallax value, and s is the point on the left figure, and s ' is the point on the right figure.I _cBe colored (c ∈ { R, G, intensity B}) of c; Calculate the normalization coupling cost on this coincidence zone $\stackrel{\‾}{E}\left(p\right)=\frac{1}{\left|\right|U_d\left(p\right)\left|\right|}{E}_d\left(p\right)=\frac{1}{\left|\right|U_d\left(p\right)\left|\right|}\underset{s\∈U_d\left(p\right)}{\mathrm{\Σ}}{e}_d\left(s\right);$ U wherein _d(p)=(x, y) | (x, y) ∈ U (p), (x-d, y) ∈ U ' (p ') }, represent adaptive region U (p) and right figure point p ' on the left figure point p adaptive region U ' (p ') overlap the zone, x in the formula, y are the horizontal ordinate of pixel in image, d is a parallax;

   || U _d(p) || be regional U _d(p) the pixel number that comprises;

   In like manner obtain with right side figure is the matching result of benchmark

   2-4) choose the minimum parallax of normalization coupling cost optimal solution the most, obtain initial parallax

   Each pixel p of left figure so that minimum parallax d is an optimal solution, is obtained initial parallax; In like manner; If the pixel p ' among the right figure obtains initial parallax so that

   

   minimum parallax d is an optimal solution.
3. 3. based on the solid matching method of the image segmentation algorithm of graph theory, it is characterized in that in the three-dimensional television system as claimed in claim 1, in said the 3rd step, following based on the image segmentation algorithm step of graph theory:

   3-1) limit collection E is arrived π=(o by the sequence arrangement that weights increase progressively ₁..., o _m);

   3-2) make that initial segmentation is S ⁰, each summit among the V all is initial segmentation S ⁰In isolated component;

   3-3) for q=1 ... .m, repeating step 3-4);

   3-4) by S ^Q-1Make up S ^qThe time, make the q bar limit o among the π _qThe node that connects is v _i, v _j, i.e. o _q=(v _i, v _j), if v _i, v _jBelong to S respectively ^Q-1In different components, and w (o _q) all littler than two component internal diversities, then merge two components, otherwise, nonjoinder; Even

   

   Be S ^Q-1In comprise v _iComponent,

   

   Be to comprise v _jComponent, if satisfy

   

   And w (o _q) less than

   

   With

   

   Threshold value T in any one, then S ^Q-1In

   

   With

   

   Be merged into one-component, obtain S ^q, otherwise S ^q=S ^Q-1, i.e. nonjoinder;

   3-5) return S=S ^mThe S that returns is the carve information of figure, and it belongs to different cut zone with the summit with a series of label information sign, by the corresponding relation on pixel and summit, promptly obtains the carve information of image.

Images