CN101976455B - Color image three-dimensional reconstruction method based on three-dimensional matching - Google Patents

Abstract

The invention relates to a color image three-dimensional reconstruction method based on three-dimensional matching, comprising the following steps of: (1) simultaneously and respectively taking an image from proper angles by using two color cameras; (2) respectively calibrating the internal parameter matrixes and the external parameter matrixes of the two cameras; (3) carrying out polar line correction and image transformation according to calibrated data; (4) working out matching cost for each pixel point in the two corrected images by applying a self-adaption weight window algorithm and acquiring an initial parallax image; (5) marking the reliability coefficient of the pixel initial matching result by adopting matching cost reliability detection and left and right consistency verification; (6) carrying out color segmentation on the images through a Mean-Shift algorithm; (7) carrying out global optimization by a selective confidence propagation algorithm on the basis of color segmentation and pixel reliability classification results to obtain a final parallax image; and (8) working out the three-dimensional coordinates of actual object points on the images according to the calibrated data and the matching relation, thereby reconstructing the three-dimensional point cloud of an object.

Description

A kind of coloured image three-dimensional rebuilding method based on the solid coupling

Technical field

The invention belongs to the technique of binocular stereoscopic vision field; Relate to problem, refer more particularly to and a kind ofly use the adaptive weighting algorithm to calculate the coupling cost and obtain the method for pixel matching relationship with reconstructed object thing three-dimensional point cloud based on the selectivity confidence spread algorithm of image segmentation and pixel fiduciary level classification results based on the colored real scene shooting image of three-dimensional matching treatment.

Background technology

Technique of binocular stereoscopic vision is a kind of method for three-dimensional measurement of passive type, and its implementation is flexible, low to environmental requirement, man-machine interaction is friendly, is a kind of technology popular in the three-dimensional reconstruction algorithm.Binocular stereo vision is intended to the mechanism of simulating human eyes identification scene three-dimensional information; Obtain the two dimensional image of scene from two angles, according to the matching relationship reconstruction of three-dimensional model of setting up between the image, mainly comprise camera calibration again; Image is to coupling, and three-dimensional information reduction etc. are process mainly.The process of setting up two width of cloth image pixel point correspondences is exactly the process of three-dimensional coupling, and it is the core of technique of binocular stereoscopic vision.

The main task of three-dimensional coupling is to obtain smooth dense disparity map true to nature; Stereo Matching Algorithm mainly is divided into local algorithm and Global Algorithm; Local algorithm utilizes the neighborhood information of pixel to mate; Computation complexity is lower, but matching precision is not high, is easy to generate mistake at low texture, parallax discontinuity zone especially.Global Algorithm adds the flatness cost in the calculating of coupling cost, makes coupling be converted into global optimum's process of energy function, mainly contains figure and cuts algorithm, confidence spread algorithm and dynamic programming algorithm.Wherein the dynamic programming algorithm computation complexity is minimum; Fastest, but be easy to generate strip flaw problem, it is higher that confidence spread algorithm and figure cut the algorithmic match precision; The disparity map edge region that calculates and the effect of degree of depth discontinuity zone are better; Comparatively speaking, it is consuming time for a long time that figure cuts algorithm, and real-time performance remains to be improved.

The existing shortcoming that has the following aspects based on the three-dimensional reconstruction algorithm of binocular stereo vision:

(1) making up suitable neighborhood window is the key of local algorithm; Window is too little, then can't comprise the enough neighborhood informations of pixel to be matched, and window is too big; Then mate in the calculating of cost and will comprise neighborhood information, the generation of these couplings that all can lead to errors with directive significance.

(2) the relatively low dynamic programming algorithm of computation complexity is limited to global energy optimization in the Global Algorithm

In the sweep trace of one dimension; Lost the slickness constraint of other directions; It is consuming time more of a specified duration that figure cuts algorithm, is difficult to satisfy the real-time requirement of real scene shooting 3-dimensional reconstruction, and the confidence spread algorithm is propagated degree of confidence indiscriminately between neighborhood territory pixel; And between parallax discontinuity zone neighborhood territory pixel, possibly not satisfy the parallax continuity constraint, the result causes reconstruction point cloud obscure boundary clear.

Owing to there is above shortcoming, existing three-dimensional reconstruction algorithm based on the solid coupling can not obtain gratifying result in practical application.

Summary of the invention

The purpose of this invention is to provide a kind of coloured image three-dimensional rebuilding method based on the solid coupling, can accurate fast automatic reconstructed image three-dimensional point cloud.

The technical scheme that the present invention adopts is: at first gather the colored real scene shooting image of two width of cloth; Carry out camera calibration; Carry out polar curve is proofreaied and correct and image transformation according to nominal data; Calculate coupling cost and initial parallax figure through initial matching, utilize confidence level detection of coupling cost and left and right sides consistency desired result that the initial matching result is classified according to fiduciary level, then the left image after proofreading and correct is carried out colour and cut apart; Carry out global optimization and obtain final parallax to have optionally the confidence spread algorithm again, utilize nominal data and matching result reconstruction of three-dimensional point cloud at last and show.

Method of the present invention specifically comprises following step:

Step 1: Image Acquisition

Use two colour TV cameras from two angles that are more or less the same same scene to be taken two width of cloth images simultaneously, what wherein the video camera on the left side was taken is the original left image, and what the video camera on the right was taken is the original right image;

Step 2: camera calibration

Respectively two video cameras are demarcated, set up the relation between camera review location of pixels and the scene location, obtain the intrinsic parameter matrix A of the video camera on the left side _L, the right the intrinsic parameter matrix A of video camera _ROuter parameter matrix [R with the video camera on the left side _Lt _L], the outer parameter matrix [R of the video camera on the right _Rt _R];

Step 3: image is proofreaied and correct polar curve

The camera interior and exterior parameter utilization method for correcting polar line that obtains according to step 2 carries out polar curve to captured left and right sides image to be proofreaied and correct and obtains run-in index binocular vision model, makes matched pixel to having identical ordinate, and left image and right image after the correction are designated as I respectively _lAnd I _r

Step 4: initial matching:

Step 4.1: confirm candidate's disparity range D:

D＝(d _min，d _max)，

D wherein _MinBe minimum parallax, d _Min=0, d _MaxBe maximum disparity, through the matched pixel point between mark benchmark image and the registering images to trying to achieve:

Ten pixels in the picked at random benchmark image pl1, and pl2, pl3 ..., pl10} seeks and { pl1 respectively in registering images; Pl2, pl3 ..., pl10} has ten estimation matched pixel points { pr1, pr2, pr3 of identical ordinate and similar color information;, pr10}, so obtain ten groups the estimation matched pixel to (pl1, pr1), (pl2, pr2); (pl3, pr3) ..., (pl10, pr10) }, to the thoroughly deserve one group parallax value { d1 of each group matched pixel to the difference of the horizontal ordinate that calculates two pixels; D2, d3 ..., d10}, maximum disparity d _Max=max{d1, d2 ..., d10}+5;

Step 4.2: adaptive weighting window algorithm

With the left image I after proofreading and correct _lBe benchmark image, with the right image I after proofreading and correct _rBe registering images, adopt the adaptive weighting windowhood method that each pixel in the benchmark image is calculated the coupling cost and obtains initial left disparity map, then, with the right image I after proofreading and correct _rBe benchmark image, with the left image I after proofreading and correct _lBe registering images, adopt the adaptive weighting windowhood method that each pixel in the benchmark image is calculated the coupling cost and obtains initial right disparity map, described adaptive weighting windowhood method is:

Step 4.2.1: weight coefficient calculates

At first benchmark image is designated as I ₁, registering images is designated as I ₂, utilize color and spatial information each pixel in two width of cloth images to be calculated the weight coefficient E of all pixels in the neighborhood window then _Pq:

$E_{\mathrm{pq}}=e^{-(\mathrm{\α\Δpq}+\mathrm{\β}{\left|\right|p-q\left|\right|}_2)},$

Wherein p is the pixel in benchmark image or the registering images, and q is for pixel p center, size being the arbitrary pixel in the neighborhood window of n * n, and n=35, Δ pq are illustrated in the color distortion between the pixel p and q under the rgb space, || p-q|| ₂Be two pixels Euclidean distances before, α and β are constant coefficient, α=0.1, β=0.047;

Step 4.2.2: the coupling cost is calculated

Under horizontal polar curve constraint, to the corresponding coupling cost C (p of all parallax value in each the pixel calculated candidate disparity range in the benchmark image ₁, d):

$C(p_1,d)=\frac{\underset{(q_1,q_2)\∈W_{p_1}\×W_{p_2}}{\mathrm{\Σ}}{E}_{p_1{q}_1}\×E_{p_2{q}_2}\×S(q_1,q_2)}{\underset{(q_1,q_2)\∈W_{p_1}\×W_{p_2}}{\mathrm{\Σ}}{E}_{p_1{q}_1}\×E_{p_2{q}_2}},$

P wherein ₁Be arbitrary pixel in the benchmark image, p ₁Coordinate do

D is the arbitrary parallax value in candidate's disparity range D, pixel p ₂Be p ₁In registering images corresponding to the candidate matches pixel of parallax d, when benchmark image is left image, p ₂Coordinate be When benchmark image is right image, p ₂Coordinate be

Represent respectively with pixel p ₁, p ₂For center, size are the neighborhood window of n * n, pixel q ₁Be window

Interior arbitrary neighborhood territory pixel point, coordinate does

Pixel q ₂Be window

In and q ₁Corresponding pixel,, when benchmark image is left image, q ₂Coordinate be When benchmark image is right image, q ₂Coordinate be With

Be the weight coefficient of trying to achieve, S (q according to step 4.2.1 ₁, q ₂) be that respective pixel is to (q ₁, q ₂) diversity factor;

Step 4.2.3: calculate the initial parallax value

Each pixel is calculated the minimum parallax value d of coupling cost ⁰(p ₁):

P wherein ₁Be the arbitrary pixel in the benchmark image, D is candidate's disparity range, d _MinAnd d _MaxBe minimum parallax and maximum disparity, C (p ₁, d) be the coupling cost that calculates according to step 4.2.1; The minimum parallax value d of coupling cost ⁰(p ₁) be pixel p ₁Initial matching parallax result;

Step 4.2.4: set up the initial parallax image

Set up initial parallax image D ⁰: D ⁰(i, j)=d ⁰(p _Ij), wherein i and j are respectively the horizontal ordinate and the ordinate of anaglyph pixel, p _IjBe that coordinate is (i, pixel j), d in the benchmark image ⁰(p _Ij) be the p that calculates among the step 4.2.3 _IjInitial matching parallax result;

If benchmark image is left image I _l, with initial parallax figure D ⁰Assignment is given initial left disparity map D _l ⁰If benchmark image is right image I _r, with initial parallax figure D ⁰Assignment is given initial right disparity map D _r ⁰

Step 5: pixel fiduciary level mark:

Step 5.1: coupling cost confidence level check

With left image I _lAll pixels are according to the classification of coupling cost confidence level, and the higher set of confidence level is designated as M _Hc, the lower set of confidence level is for being designated as M _Lc: left image I _lIn arbitrary pixel p _lCoupling cost confidence level is r (p _l):

C _Min1Be p _lThe coupling cost that initial matching parallax result is corresponding, i.e. smallest match cost value, and C _Min2Be p _lThe second little coupling cost, setting threshold dist then is as r (p _L1During)＞dist, p _lThe matching result confidence level be higher, p _l∈ M _Hc, otherwise confidence level is for lower, p _l∈ M _Lc, wherein threshold value dist gets 0.04;

Step 5.2: left and right sides consistency desired result

For the arbitrary pixel p in the left image _l, coordinate does

p _lThe initial parallax result The matched pixel p of correspondence in right image _rCoordinate do The initial right anaglyph D that obtains according to step 4 _r ⁰Obtain pixel p _rThe initial parallax result

If d ₁=d ₂, pixel p then _lThrough left and right sides consistency desired result, be designated as p _l∈ M _Ac, otherwise, pixel p _l, be not designated as p through left and right sides consistency desired result _l∈ M _Bc, M wherein _AcAnd M _BcBe respectively through the set of left and right sides consistency desired result and the set through left and right sides consistency desired result;

Step 5.3: pixel fiduciary level coefficient mark

According to step 5.1 and 5.2 result to each element marking fiduciary level coefficient Con (p in the left image _l):

$\mathrm{Con}\left(p_l\right)=\left\{\begin{array}{ccc}4,& \mathrm{if}& p_l\∈M_{\mathrm{hc}}\∩M_{\mathrm{ac}}\\ 3,& \mathrm{if}& p_l\∈M_{\mathrm{lc}}\∩M_{\mathrm{ac}}\\ 2,& \mathrm{if}& p_l\∈M_{\mathrm{hc}}\∩M_{\mathrm{bc}}\\ 1,& \mathrm{if}& p_l\∈M_{\mathrm{lc}}\∩M_{\mathrm{bc}}\end{array}\right.$

P wherein _lBe the arbitrary pixel in the left image, Con (p _l) be p _lThe fiduciary level coefficient;

Step 6: image segmentation:

With the Mean-Shift algorithm left image is cut apart, to the cut zone S (p under each element marking _l), p wherein _lBe arbitrary pixel in the left image, S (p _l) be pixel p _lAffiliated region labeling;

Step 7: global optimization

Step 7.1: the level and smooth cost of pixel is calculated

Calculate the level and smooth cost J (p between four neighborhood territory pixels up and down of each pixel and this pixel in the left image with respect to all parallax value in the scope of the inspection D _l, q _l, d _p, d _q):

J(p _l，q _l，d _p，d _q)＝min{|d _p-d _q|，|d _max-d _min|/8}，

P wherein _lBe the arbitrary pixel in the left image, q _lBe pixel p _lArbitrary neighbours territory pixel, d _pAnd d _qBe respectively pixel p _lAnd q _lThe arbitrary parallax in disparity range D, d _MaxAnd d _MinBe maximum disparity and minimum parallax;

Step 7.2: the degree of confidence message of calculating pixel node

Iterative computation degree of confidence message, t is a number of iterations, and initial value is 0, when t=50, stops iteration, and the computation process of iteration is each time:

During t iteration; This pixel was propagated to neighbours territory pixel when each pixel node in the left image was calculated next iteration, with respect to the degree of confidence message

of each parallax value in the disparity range D

${M_{p_l{q}_l}}^t\left(d\right)=\underset{d_x\∈D}{\min }(C(p_l,d)+\underset{q_s\∈N_1\left(p_l\right)\backslash q_l}{\mathrm{\Σ}}{M_{q_s{p}_l}}^{t-1}\left(d_x\right)+J(p_l,q_l,{d,d}_x)),$

P wherein _lBe arbitrary pixel in the left image, q _lBe pixel p _lAny neighbours territory pixel, D is the disparity range of definition in the step 4.1, d is the arbitrary parallax value in the D, C (p _l, d) coupling cost, d for calculating among the step 4.2.2 _xBe the arbitrary parallax value in the disparity range D, J (p _l, q _l, d, d _x) the level and smooth cost of trying to achieve for step 7.1,

For t-1 iteration try to achieve from pixel q _sTo p _lThe parallax of propagating is d _xDegree of confidence message, during t=1

Be 0, d _xBe the arbitrary parallax value in the disparity range D, q _sBe pixel p _lSelectivity neighborhood N ₁(p _l) in, be different from pixel q _lArbitrary pixel, described selectivity neighborhood N ₁(p _l) be:

N ₁(p _l)＝{q _f|q _f∈N(p _l)，Con(q _f)≥Con(p _l)and?S(q _f)＝S(p _l)}，

N in the formula ₁(p _l) be pixel p _lThe territory of neighbours up and down, Con (q _f) and Con (p _l) be the fiduciary level coefficient of mark in the step 5.3, S (q _f) and S (p _l) be the pixel q that tries to achieve in the step 6 _fAnd p _lAffiliated cut zone label;

Step 7.3: calculate each pixel in the left image with respect to might parallax degree of confidence b (p _l, d):

P wherein _lBe the arbitrary pixel in the left image, d is the arbitrary parallax value in the D, C (p _l, the coupling cost that d) obtains for step 4.2.2,

Be the 50th iterative computation obtain from pixel p _sTo p _lThe parallax of propagating is the degree of confidence message of d, p _sBe N ₁(p _l) interior arbitrary pixel, N ₁(p _l) be the p of definition in the step 7.2 _lThe selectivity neighborhood;

Step 7.4: calculate anaglyph

The optimum parallax value d of confidence calculations (p according to each pixel _l):

$d\left(p_l\right)=\underset{d\∈D}{\arg \min }b(p_l,d),$

P wherein _lBe the arbitrary pixel in the left image, b (p _l, the degree of confidence that d) calculates for step 7.3, D is a disparity range, d is the arbitrary parallax value in the inspection scope D;

Optimum parallax according to each pixel in the left image is set up final parallax as D ^Out: D ^Out(x, y)=d (p _Xy), wherein x and y are respectively anaglyph D ^OutThe horizontal ordinate of pixel and ordinate, p _XyBe that coordinate is (x, pixel y), d (p in the benchmark image _Xy) be p _XyOptimum parallax value;

Step 8: the three-dimensional information of reconstructed object thing

The camera interior and exterior parameter matrix A that obtains according to step 2 _L, A _R[R _Lt _L], [R _Rt _R], and the disparity map D that obtains of step 7 ^Out, calculate the three-dimensional point cloud model of whole object through the space method of crossing.Beneficial effect: compared with prior art; The present invention has following advantage: the adaptive weighting window algorithm calculates its weight with respect to pixel to be matched according to the space and the colouring information of neighborhood territory pixel, has avoided the intrinsic self-adapting window building process of difficulty of local algorithm; The tradition certainty factor algebra propagates degree of confidence message between all neighbors; Do not have directive significance owing to do not satisfy the initial matching result of parallax continuity constraint and some pixel between the part neighbor; There is irrational travel path among the tradition certainty factor algebra; Cause that matching accuracy is not high, optimal speed waits problem slowly, the present invention utilizes color images and pixel fiduciary level classification results to instruct the scope and the direction of degree of confidence message propagation, and this have optionally that the confidence spread algorithm has cut off irrational part among traditional certainty factor algebra; Make the path of global energy optimization be optimized; Computation complexity reduces and has more specific aim, and the matching result of low fiduciary level pixel constantly obtains revising in the process of iteration optimization, finally obtains the higher disparity map of matching precision.The present invention has fully combined the advantage of local optimum algorithm and global optimum's algorithm, and the two is combined, and has overcome existing three-dimensional reconstruction technology in the contradiction of rebuilding between accuracy and the reconstruction speed, and has improved the automaticity of process of reconstruction.

Description of drawings

Fig. 1 is entire flow figure of the present invention.

Fig. 2 is the process flow diagram of the adaptive weighting window matching algorithm that adopts in step 4 initial matching of the present invention.

Fig. 3 is the process flow diagram of step 5 pixel fiduciary level labeling algorithm of the present invention.

Fig. 4 is the optionally confidence spread algorithm flow chart that has that adopts in the step 7 of the present invention.

Fig. 5 system model and principle schematic.

Fig. 6 polar curve is proofreaied and correct synoptic diagram.

Fig. 7 adaptive weighting window synoptic diagram.

Fig. 8 respective pixel diversity factor is calculated synoptic diagram.

Fig. 9 pixel fiduciary level classification synoptic diagram.

Figure 10 tradition confidence spread algorithm travel path synoptic diagram.

Figure 11 is based on the confidence spread path synoptic diagram of pixel fiduciary level classification.

Figure 12 is based on the confidence spread path synoptic diagram of image segmentation result.

Figure 13 is calculated the 3 d space coordinate synoptic diagram of object point on the picture by matching relationship and nominal data.

Embodiment

With reference to the accompanying drawings, specific embodiments of the present invention is done more detailed description.The programming implementation tool is selected Visual C++6.0 and OpenCV Flame Image Process function library for use, has taken the coloured image that two width of cloth contain the discontinuous and low texture region of more parallax in the indoor environment.

Fig. 1 is entire flow figure of the present invention

Fig. 5 is system model of the present invention and principle schematic.Use two colored CCDs to take a width of cloth coloured image, O simultaneously from two different angles respectively _L, O _RBe respectively the photocentre of two video cameras, I _L, I _RBe respectively the imaging plane of two video cameras, P is a space object point of treating on the object of reconstruct, P _L, P _RBe object point P imaging point on two video camera imaging planes respectively.This is a pair of match point by the same space object point imaging point on the different cameras imaging plane.Appoint and to get wherein that a width of cloth is a benchmark image, another width of cloth is a registering images, and the process of in alignment image, search for corresponding match point for each pixel in the benchmark image is called three-dimensional the coupling.After obtaining the matching relationship of pixel,,, carry out reverse computing, just can obtain the 3 d space coordinate of corresponding object point, thereby realize the three-dimensionalreconstruction of image in conjunction with demarcating the camera interior and exterior parameter that obtains according to system model.

Fig. 6 proofreaies and correct synoptic diagram for polar curve.For pixel p among the left figure _l, matched pixel p _rSearch only need in right figure corresponding to p _lPolar curve on carry out, and all polar curves of parallel stereovision model all are parallel to the line O of photocentre _lO _r, then stereo-picture can further reduce the search difficulty in the case to having only horizontal shift, and the search of corresponding point only gets final product along same line search.But in reality, this master pattern is difficult to satisfy, and imaging plane can be proofreaied and correct through polar curve and make the imaging plane rotation, thereby obtain two virtual parallel imaging planes not on same plane.Through rotating initial projection matrix around photocentre up to two focal plane coplanes, baseline is also contained in the focal plane, thereby obtains two new projection matrixes.Limit is located in infinite distant place like this; Therefore polar curve is parallel.In order to make that simultaneously polar curve is a level, baseline must be parallel to the new X-direction of two cameras.In addition, in order to obtain correct correction, conjugate points is to having identical ordinate, and this can obtain through letting new camera configuration have identical intrinsic parameter.

Method of the present invention specifically comprises following step:

Step 1: Image Acquisition

Use two colour TV cameras from two angles that are more or less the same same scene to be taken two width of cloth images simultaneously, what wherein the video camera on the left side was taken is the original left image, and what the video camera on the right was taken is the original right image;

Step 2: camera calibration

Respectively two video cameras are demarcated, set up the relation between camera review location of pixels and the scene location, obtain the intrinsic parameter matrix A of the video camera on the left side _L, the right the intrinsic parameter matrix A of video camera _ROuter parameter matrix [R with the video camera on the left side _Lt _L], the outer parameter matrix [R of the video camera on the right _Rt _R];

The camera calibration technology is comparative maturity now; List of references " A Flexible New Technique for Camera Calibration " (Zhang Z Y; IEEE Transactions on Pattern Analysis and Machine Intelligence; 2000,20 (11): 1330-1334) proposed a kind of calibration algorithm that is called the plane template method, adopted this method respectively two video cameras to be demarcated among the present invention;

Step 3: image is proofreaied and correct polar curve

The camera interior and exterior parameter utilization method for correcting polar line that obtains according to step 2 carries out polar curve to captured left and right sides image to be proofreaied and correct and obtains run-in index binocular vision model, makes matched pixel to having identical ordinate, and left image and right image after the correction are designated as I respectively _lAnd I _r

Adopt list of references " A compact algorithm for rectification of stereo pairs.Machine Vision and Applications " (Fusiello A; Trucco E; Verri A.2000,12 (1): the method for correcting polar line of the proposition 16-22) carries out polar curve is proofreaied and correct to captured left and right sides image, and is as shown in Figure 6; When if the pixel coordinate after the conversion in the image corresponds on the non-integer coordinates in the original image; Then carry out bilinear interpolation, obtain run-in index binocular vision model at last, the image after overcorrect is undistorted; Error between the right ordinate of matched pixel is less than a pixel, and the space complexity of coupling reduces greatly.

Step 4: initial matching:

Step 4.1: confirm candidate's disparity range D:

D＝(d _min，d _max)，

D wherein _MinBe minimum parallax, d _Min=0, d _MaxBe maximum disparity, through the matched pixel point between mark benchmark image and the registering images to trying to achieve:

Ten pixels in the picked at random benchmark image pl1, and pl2, pl3 ..., pl10} seeks and { pl1 respectively in registering images; Pl2, pl3 ..., pl10} has ten estimation matched pixel points { pr1, pr2, pr3 of identical ordinate and similar color information;, pr10}, so obtain ten groups the estimation matched pixel to (pl1, pr1), (pl2, pr2); (pl3, pr3) ..., (pl10, pr10) }, to the thoroughly deserve one group parallax value { d1 of each group matched pixel to the difference of the horizontal ordinate that calculates two pixels; D2, d3 ..., d10}, maximum disparity d _Max=max{d1, d2 ..., d10}+5;

Step 4.2: adaptive weighting window algorithm

With the left image I after proofreading and correct _lBe benchmark image, with the right image I after proofreading and correct _rBe registering images, adopt the adaptive weighting windowhood method that each pixel in the benchmark image is calculated the coupling cost and obtains initial left disparity map, then, with the right image I after proofreading and correct _rBe benchmark image, with the left image I after proofreading and correct _lBe registering images, adopt the adaptive weighting windowhood method that each pixel in the benchmark image is calculated the coupling cost and obtains initial right disparity map, described adaptive weighting windowhood method is:

Step 4.2.1: weight coefficient calculates

At first benchmark image is designated as I ₁, registering images is designated as I ₂, utilize color and spatial information each pixel in two width of cloth images to be calculated the weight coefficient E of all pixels in the neighborhood window then _Pq:

$E_{\mathrm{pq}}=e^{-(\mathrm{\α\Δpq}+\mathrm{\β}{\left|\right|p-q\left|\right|}_2)},$

Wherein p is the pixel in benchmark image or the registering images, and q is for pixel p center, size being the arbitrary pixel in the neighborhood window of n * n, and n=35, Δ pq are illustrated in the color distortion between the pixel p and q under the rgb space,

The r of c presentation video wherein, g or b passage, I _c(p) and I _c(q) remarked pixel p and the q color component under the c passage, || p-q|| ₂Be two pixels Euclidean distances before, α and β are constant coefficient, α=0.1, β=0.047;

Step 4.2.2: the coupling cost is calculated

Under horizontal polar curve constraint, to the corresponding coupling cost C (p of all parallax value in each the pixel calculated candidate disparity range in the benchmark image ₁, d):

$C(p_1,d)=\frac{\underset{(q_1,q_2)\∈W_{p_1}\×W_{p_2}}{\mathrm{\Σ}}{E}_{p_1{q}_1}\×E_{p_2{q}_2}\×S(q_1,q_2)}{\underset{(q_1,q_2)\∈W_{p_1}\×W_{p_2}}{\mathrm{\Σ}}{E}_{p_1{q}_1}\×E_{p_2{q}_2}},$

P wherein ₁Be arbitrary pixel in the benchmark image, p ₁Coordinate do

D is the arbitrary parallax value in candidate's disparity range D, pixel p ₂Be p ₁In registering images corresponding to the candidate matches pixel of parallax d, when benchmark image is left image, p ₂Coordinate be

When benchmark image is right image, p ₂Coordinate be

Represent respectively with pixel p ₁, p ₂For center, size are the neighborhood window of n * n, pixel q ₁Be window Interior arbitrary neighborhood territory pixel point, coordinate does

Pixel q ₂Be window

In and q ₁Corresponding pixel,, when benchmark image is left image, q ₂Coordinate be

When benchmark image is right image, q ₂Coordinate be

With

Be the weight coefficient of trying to achieve, S (q according to step 4.2.1 ₁, q ₂) be that respective pixel is to (q ₁, q ₂) diversity factor;

It is as shown in Figure 8,

Wherein, q _2lBe q ₂Left neighborhood territory pixel, coordinate does q _2rBe q ₂Right neighborhood territory pixel, coordinate does

I ₂(q ₂), I ₂(q _2l) and I ₂(q _2r) be respectively pixel q ₂, q _2lAnd q _2rAt registering images I ₂In the mean value of RGB triple channel component, definition then,

Pixel q then ₁And q ₂Diversity factor be:

S＝max{0，I ₁(q ₁)-I _max，I _min-I ₁(q ₁)}，

I wherein ₁(q ₁) be pixel q ₁At benchmark image I ₁In the mean value of RGB triple channel component;

Step 4.2.3: calculate the initial parallax value

Each pixel is calculated the minimum parallax value d of coupling cost ⁰(p ₁):

P wherein ₁Be the arbitrary pixel in the benchmark image, D is candidate's disparity range, d _MinAnd d _MaxBe minimum parallax and maximum disparity, C (p ₁, d) be the coupling cost that calculates according to step 4.2.1; The minimum parallax value d of coupling cost ⁰(p ₁) be pixel p ₁Initial matching parallax result;

Step 4.2.4: set up the initial parallax image

Set up initial parallax image D ⁰: D ⁰(i, j)=d ⁰(p _Ij), wherein i and j are respectively the horizontal ordinate and the ordinate of anaglyph pixel, p _IjBe that coordinate is (i, pixel j), d in the benchmark image ⁰(p _Ij) be the p that calculates among the step 4.2.3 _IjInitial matching parallax result;

If benchmark image is left image I _l, with initial parallax figure D ⁰Assignment is given initial left disparity map D _l ⁰If benchmark image is right image I _r, with initial parallax figure D ⁰Assignment is given initial right disparity map D _r ⁰

Step 5: pixel fiduciary level mark:

Step 5.1: coupling cost confidence level check

With left image I _lAll pixels are according to the classification of coupling cost confidence level, and the higher set of confidence level is designated as M _Hc, the lower set of confidence level is for being designated as M _Lc: left image I _lIn arbitrary pixel p _lCoupling cost confidence level is r (p _l):

C _Min1Be p _lThe coupling cost that initial matching parallax result is corresponding, i.e. smallest match cost value, and C _Min2Be p _lThe second little coupling cost, setting threshold dist then is as r (p _L1During)＞dist, p _lThe matching result confidence level be higher, p _l∈ M _Hc, otherwise confidence level is for lower, p _l∈ M _Lc, wherein threshold value dist gets 0.04;

Step 5.2: left and right sides consistency desired result

For the arbitrary pixel p in the left image _l, coordinate does

p _lThe initial parallax result

The matched pixel p of correspondence in right image _rCoordinate do The initial right anaglyph D that obtains according to step 4 _r ⁰Obtain pixel p _rThe initial parallax result

If d ₁=d ₂, pixel p then _lThrough left and right sides consistency desired result, be designated as p _l∈ M _Ac, otherwise, pixel p _l, be not designated as p through left and right sides consistency desired result _l∈ M _Bc, M wherein _AcAnd M _BcBe respectively through the set of left and right sides consistency desired result and the set through left and right sides consistency desired result;

Step 5.3: pixel fiduciary level coefficient mark

According to step 5.1 and 5.2 result to each element marking fiduciary level coefficient Con (p in the left image _l):

$\mathrm{Con}\left(p_l\right)=\left\{\begin{array}{ccc}4,& \mathrm{if}& p_l\∈M_{\mathrm{hc}}\∩M_{\mathrm{ac}}\\ 3,& \mathrm{if}& p_l\∈M_{\mathrm{lc}}\∩M_{\mathrm{ac}}\\ 2,& \mathrm{if}& p_l\∈M_{\mathrm{hc}}\∩M_{\mathrm{bc}}\\ 1,& \mathrm{if}& p_l\∈M_{\mathrm{lc}}\∩M_{\mathrm{bc}}\end{array}\right.$

P wherein _lBe the arbitrary pixel in the left image, Con (p _l) be p _lThe fiduciary level coefficient;

Step 6: image segmentation:

With the Mean-Shift algorithm left image is cut apart, to the cut zone S (p under each element marking _l), p wherein _lBe arbitrary pixel in the left image, S (p _l) be pixel p _lAffiliated region labeling;

Parameter is set to: spatial bandwidth h _s=7, color bandwidth h _r=6.5, smallest region size M=35;

Step 7: global optimization

Traditional confidence spread path is shown in figure 10, pixel p ₀Obtain degree of confidence from four neighborhood territory pixels, solid arrow is represented the direction of propagation among the figure, and dotted arrow is represented the direction of last round of confidence spread.Suppose pixel p ₀₁Be insecure pixel, so in the confidence spread process, from p ₀₁Information M _P01p0Fiduciary level also just relatively low, thereby pixel p ₀The coupling cost added insecure information, the matching result that possibly lead to errors in calculating.To this problem; The present invention is on the basis of pixel fiduciary level classification; Improved the path of traditional confidence spread; Shown in figure 11; Fiduciary level with four kinds of different pattern remarked pixels; The fiduciary level of

remarked pixel is the highest; The fiduciary level coefficient is 4; remarked pixel fiduciary level coefficient is 3,

remarked pixel fiduciary level coefficient be 2,

remarked pixel fiduciary level is minimum; The fiduciary level coefficient is 1; We are defined in when propagating degree of confidence between the neighbor, if two pixel fiduciary levels do not wait, the direction of propagation is low by the high sensing fiduciary level of fiduciary level; If two pixel fiduciary levels equate, take the principle of two-way propagation.Match information flows to insecure network from the reliable network that approaches true parallax makes the global optimization approach of this paper have selectivity in the direction of propagation.

The parallax continuity is the prerequisite of confidence spread, and is shown in figure 10, pixel p ₀₂With pixel p ₀The both sides and the true parallax that lay respectively at the object edge differ greatly, so from p ₀₂The degree of confidence of propagating and coming is to pixel p ₀To not have directive significance.Yet have the discontinuous zone of many degree of depth in the three-dimensional scenic, it is unaccommodated in these zones, propagating degree of confidence.The zone of parallax saltus step often is accompanied by change in color, and based on this fact, the present invention utilizes colored carve information to retrain the scope of confidence spread, avoids the regional spread degree of confidence in color generation saltus step, and is shown in figure 12, s ₁And s ₂Represent two different cut zone respectively, be defined in and propagate degree of confidence in the identical block that the travel path that belongs between two pixels of different blocks breaks off.This algorithm that receives carve information constraint can reduce degree of confidence effectively to be propagated between parallax differs bigger neighbor, has improved the matching performance of BP algorithm at the parallax discontinuity zone.

Step 7.1: the level and smooth cost of pixel is calculated

Calculate the level and smooth cost J (p between four neighborhood territory pixels up and down of each pixel and this pixel in the left image with respect to all parallax value in the scope of the inspection D _l, q _l, d _p, d _q):

J(p _l，q _l，d _p，d _q)＝min{|d _p-d _q|，|d _max-d _min|/8}，

P wherein _lBe the arbitrary pixel in the left image, q _lBe pixel p _lArbitrary neighbours territory pixel, d _pAnd d _qBe respectively pixel p _lAnd q _lThe arbitrary parallax in disparity range D, d _MaxAnd d _MinBe maximum disparity and minimum parallax;

Step 7.2: the degree of confidence message of calculating pixel node

Iterative computation degree of confidence message, t is a number of iterations, and initial value is 0, when t=50, stops iteration, and the computation process of iteration is each time:

During t iteration; This pixel was propagated to neighbours territory pixel when each pixel node in the left image was calculated next iteration, with respect to the degree of confidence message

of each parallax value in the disparity range D

${M_{p_l{q}_l}}^t\left(d\right)=\underset{d_x\∈D}{\min }(C(p_l,d)+\underset{q_s\∈N_1\left(p_l\right)\backslash q_l}{\mathrm{\Σ}}{M_{q_s{p}_l}}^{t-1}\left(d_x\right)+J(p_l,q_l,{d,d}_x)),$

P wherein _lBe arbitrary pixel in the left image, q _lBe pixel p _lAny neighbours territory pixel, D is the disparity range of definition in the step 4.1, d is the arbitrary parallax value in the D, C (p _l, d) coupling cost, d for calculating among the step 4.2.2 _xBe the arbitrary parallax value in the disparity range D, J (p _l, q _l, d, d _x) the level and smooth cost of trying to achieve for step 7.1,

For t-1 iteration try to achieve from pixel q _sTo p _lThe parallax of propagating is d _xDegree of confidence message, during t=1

Be 0, d _xBe the arbitrary parallax value in the disparity range D, q _sBe pixel p _lSelectivity neighborhood N ₁(p _l) in, be different from pixel q _lArbitrary pixel, described selectivity neighborhood N ₁(p _l) be:

N ₁(p _l)＝{q _f|q _f∈N(p _l)，Con(q _f)≥Con(p _l)and?S(q _f)＝S(p _l)}，

N in the formula ₁(p _l) be pixel p _lThe territory of neighbours up and down, Con (q _f) and Con (p _l) be the fiduciary level coefficient of mark in the step 5.3, S (q _f) and S (p _l) be the pixel q that tries to achieve in the step 6 _fAnd p _lAffiliated cut zone label;

Step 7.3: calculate each pixel in the left image with respect to might parallax degree of confidence b (p _l, d):

P wherein _lBe the arbitrary pixel in the left image, d is the arbitrary parallax value in the D, C (p _l, the coupling cost that d) obtains for step 4.2.2,

Be the 50th iterative computation obtain from pixel p _sTo p _lThe parallax of propagating is the degree of confidence message of d, p _sBe N ₁(p _l) interior arbitrary pixel, N ₁(p _l) be the p of definition in the step 7.2 _lThe selectivity neighborhood;

Step 7.4: calculate anaglyph

The optimum parallax value d of confidence calculations (p according to each pixel _l):

$d\left(p_l\right)=\underset{d\∈D}{\arg \min }b(p_l,d),$

P wherein _lBe the arbitrary pixel in the left image, b (p _l, the degree of confidence that d) calculates for step 7.3, D is a disparity range, d is the arbitrary parallax value in the inspection scope D;

Optimum parallax according to each pixel in the left image is set up final parallax as D ^Out: D ^Out(x, y)=d (p _Xy), wherein x and y are respectively anaglyph D ^OutThe horizontal ordinate of pixel and ordinate, p _XyBe that coordinate is (x, pixel y), d (p in the benchmark image _Xy) be p _XyOptimum parallax value;

Step 8: the three-dimensional information of reconstructed object thing

The camera interior and exterior parameter matrix A that obtains according to step 2 _L, A _R[R _Lt _L], [R _Rt _R], and the disparity map D that obtains of step 7 ^Out, calculate the three-dimensional point cloud model of whole object through the space method of crossing.

Figure 13 is the cross synoptic diagram of method of space, O _L, O _RBe respectively the photocentre of two video cameras, S _L, S _RBe respectively the imaging plane of two video cameras, P _L, P _RIt is a pair of match point in two shot by camera images.Following relation of plane is arranged between the pixel coordinate on object point three-dimensional coordinate and the imaging plane in the space:

$s\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=A\left[\begin{array}{cc}R& T\\ 0^T& 1\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]$

Wherein (u, the v) pixel coordinate of representation space object point imaging vegetarian refreshments on imaging plane, (X _w, Y _w, Z _w) expression object point volume coordinate.It has represented a straight-line equation through video camera photocentre, imaging point, space object point.

Arbitrary pixel p in the image of a left side _lMatched pixel point in right figure is p _r, p _rCoordinate do

Wherein

The optimum anaglyph that calculates for step 7.4,

Be p _lCoordinate, therefore as long as according to match point to { p _l, p _rCoordinate just can calculate two straight-line equations that project a pair of matched pixel point on two imaging planes through same object point; Calculate the intersection point of two straight lines; Just can obtain the 3 d space coordinate of object point; Because all have error in each processes such as demarcation, coupling, computing, reverse two straight lines that calculate can just in time not intersect probably, then get the mid point of its common vertical line this moment.

Claims (1)

1. coloured image three-dimensional rebuilding method based on solid coupling is characterized in that this method for reconstructing contains following steps successively:

Step 1: Image Acquisition

Use two colour TV cameras from two angles that are more or less the same same scene to be taken two width of cloth images simultaneously, what wherein the video camera on the left side was taken is the original left image, and what the video camera on the right was taken is the original right image;

Step 2: camera calibration

Respectively two video cameras are demarcated, set up the relation between camera review location of pixels and the scene location, obtain the intrinsic parameter matrix A of the video camera on the left side _L, the right the intrinsic parameter matrix A of video camera _ROuter parameter matrix [R with the video camera on the left side _Lt _L], the outer parameter matrix [R of the video camera on the right _Rt _R];

Step 3: image is proofreaied and correct polar curve

The camera interior and exterior parameter utilization method for correcting polar line that obtains according to step 2 carries out polar curve to captured left and right sides image to be proofreaied and correct and obtains run-in index binocular vision model, makes matched pixel to having identical ordinate, and left image and right image after the correction are designated as I respectively _lAnd I _r

Step 4: initial matching:

Step 4.1: confirm candidate's disparity range D:

D＝(d _min，d _max)，

D wherein _MinBe minimum parallax, d _Min=0, d _MaxBe maximum disparity, through the matched pixel point between mark benchmark image and the registering images to trying to achieve:

Ten pixels in the picked at random benchmark image pl1, and pl2, pl3 ..., pl10} seeks and { pl1 respectively in registering images; Pl2, pl3 ..., pl10} has ten estimation matched pixel points { pr1, pr2, pr3 of identical ordinate and similar color information;, pr10}, so obtain ten groups the estimation matched pixel to (pl1, pr1), (pl2, pr2); (pl3, pr3) ..., (pl10, pr10) }, to the thoroughly deserve one group parallax value { d1 of each group matched pixel to the difference of the horizontal ordinate that calculates two pixels; D2, d3 ..., d10}, maximum disparity d _Max=max{d1, d2 ..., d10}+5;

Step 4.2: adaptive weighting window algorithm

With the left image I after proofreading and correct _lBe benchmark image, with the right image I after proofreading and correct _rBe registering images, adopt the adaptive weighting windowhood method that each pixel in the benchmark image is calculated the coupling cost and obtains initial left disparity map, then, with the right image I after proofreading and correct _rBe benchmark image, with the left image I after proofreading and correct _lBe registering images, adopt the adaptive weighting windowhood method that each pixel in the benchmark image is calculated the coupling cost and obtains initial right disparity map, described adaptive weighting windowhood method is:

Step 4.2.1: weight coefficient calculates

At first benchmark image is designated as I ₁, registering images is designated as I ₂, utilize color and spatial information each pixel in two width of cloth images to be calculated the weight coefficient E of all pixels in the neighborhood window then _Pq:

$E_{\mathrm{pq}}=e^{-(\mathrm{\α\Δpq}+\mathrm{\β}{\left|\right|p-q\left|\right|}_2)},$

Wherein p is the pixel in benchmark image or the registering images, and q is for pixel p center, size being the arbitrary pixel in the neighborhood window of n * n, and n=35, Δ pq are illustrated in the color distortion between the pixel p and q under the rgb space, || p-q|| ₂Be two pixels Euclidean distances before, α and β are constant coefficient, α=0.1, β=0.047;

Step 4.2.2: the coupling cost is calculated

Under horizontal polar curve constraint, to the corresponding coupling cost C (p of all parallax value in each the pixel calculated candidate disparity range in the benchmark image ₁, d):

$C(p_1,d)=\frac{\underset{(q_1,q_2)\∈W_{p_1}\×W_{p_2}}{\mathrm{\Σ}}{E}_{p_1{q}_1}\×E_{p_2{q}_2}\×S(q_1,q_2)}{\underset{(q_1,q_2)\∈W_{p_1}\×W_{p_2}}{\mathrm{\Σ}}{E}_{p_1{q}_1}\×E_{p_2{q}_2}},$

P wherein ₁Be arbitrary pixel in the benchmark image, p ₁Coordinate do

D is the arbitrary parallax value in candidate's disparity range D, pixel p ₂Be p ₁In registering images corresponding to the candidate matches pixel of parallax d, when benchmark image is left image, p ₂Coordinate be

When benchmark image is right image, p ₂Coordinate be

Represent respectively with pixel p ₁, p ₂For center, size are the neighborhood window of n * n, pixel q ₁Be window

Interior arbitrary neighborhood territory pixel point, coordinate does

Pixel q ₂Be window

In and q ₁Corresponding pixel, when benchmark image is left image, q ₂Coordinate be

When benchmark image is right image, q ₂Coordinate be With

Be the weight coefficient of trying to achieve, S (q according to step 4.2.1 ₁, q ₂) be that respective pixel is to (q ₁, q ₂) diversity factor;

Step 4.2.3: calculate the initial parallax value

Each pixel is calculated the minimum parallax value d of coupling cost ⁰(p ₁):

P wherein ₁Be the arbitrary pixel in the benchmark image, D is candidate's disparity range, d _MinAnd d _MaxBe minimum parallax and maximum disparity, C (p ₁, d) be the coupling cost that calculates according to step 4.2.1; The minimum parallax value d of coupling cost ⁰(p ₁) be pixel p ₁Initial matching parallax result;

Step 4.2.4: set up the initial parallax image

Set up initial parallax image D ⁰: D ⁰(i, j)=d ⁰(p _Ij), wherein i and j are respectively the horizontal ordinate and the ordinate of anaglyph pixel, p _IjBe that coordinate is (i, pixel j), d in the benchmark image ⁰(p _Ij) be the p that calculates among the step 4.2.3 _IjInitial matching parallax result;

If benchmark image is left image I _l, with initial parallax image D ⁰Assignment is given initial left disparity map D _l ⁰If benchmark image is right image I _r, with initial parallax image D ⁰Assignment is given initial right disparity map D _r ⁰

Step 5: pixel fiduciary level mark:

Step 5.1: coupling cost confidence level check

With left image I _lAll pixels are according to the classification of coupling cost confidence level, and the higher set of confidence level is designated as M _Hc, the lower set of confidence level is for being designated as M _Lc: left image I _lIn arbitrary pixel p _lCoupling cost confidence level is r (p _l):

C _Min1Be p _lThe coupling cost that initial matching parallax result is corresponding, i.e. smallest match cost value, and C _Min2Be p _lThe second little coupling cost, setting threshold dist then is as r (p _lDuring)＞dist, p _lThe matching result confidence level be higher, p _l∈ M _Hc, otherwise confidence level is for lower, p _l∈ M _Lc, wherein threshold value dist gets 0.04;

Step 5.2: left and right sides consistency desired result

For the arbitrary pixel p in the left image _l, coordinate does p _lThe initial parallax result The matched pixel p of correspondence in right image _rCoordinate do

The initial right anaglyph D that obtains according to step 4 _r ⁰Obtain pixel p _rInitial matching parallax result

If d ₁=d ₂, pixel p then _lThrough left and right sides consistency desired result, be designated as p _l∈ M _Ac, otherwise, pixel p _l, be not designated as p through left and right sides consistency desired result _l∈ M _Bc, M wherein _AcAnd M _BcBe respectively through the set of left and right sides consistency desired result and the set through left and right sides consistency desired result;

Step 5.3: pixel fiduciary level coefficient mark

According to step 5.1 and 5.2 result to each element marking fiduciary level coefficient Con (p in the left image _l):

$\mathrm{Con}\left(p_l\right)=\left\{\begin{array}{ccc}4,& \mathrm{if}& p_l\∈M_{\mathrm{hc}}\∩M_{\mathrm{ac}}\\ 3,& \mathrm{if}& p_l\∈M_{\mathrm{lc}}\∩M_{\mathrm{ac}}\\ 2,& \mathrm{if}& p_l\∈M_{\mathrm{hc}}\∩M_{\mathrm{bc}}\\ 1,& \mathrm{if}& p_l\∈M_{\mathrm{lc}}\∩M_{\mathrm{bc}}\end{array}\right.$

P wherein _lBe the arbitrary pixel in the left image, Con (p _l) be p _lThe fiduciary level coefficient;

Step 6: image segmentation:

With the Mean-Shift algorithm left image is cut apart, to the cut zone S (p under each element marking _l), p wherein _lBe arbitrary pixel in the left image, S (p _l) be pixel p _lAffiliated region labeling;

Step 7: global optimization

Step 7.1: the level and smooth cost of pixel is calculated

Calculate the level and smooth cost J (p between four neighborhood territory pixels up and down of each pixel and this pixel in the left image with respect to all parallax value in candidate's disparity range D _l, q ₁, d _p, d _q):

J(p _l，q _l，d _p，d _q)＝min{|d _p-d _q|，|d _max-d _min|/8}，

P wherein _lBe the arbitrary pixel in the left image, q _lBe pixel p _lArbitrary neighbours territory pixel, d _pAnd d _qBe respectively pixel p _lAnd q _lThe arbitrary parallax in disparity range D, d _MaxAnd d _MinBe maximum disparity and minimum parallax;

Step 7.2: the degree of confidence message of calculating pixel node

Iterative computation degree of confidence message, t is a number of iterations, and initial value is 0, when t=50, stops iteration, and the computation process of iteration is each time:

During t iteration; This pixel was propagated to neighbours territory pixel when each pixel node in the left image was calculated next iteration, with respect to the degree of confidence message

of each parallax value in candidate's disparity range D

${M_{p_l{q}_l}}^t\left(d\right)=\underset{d_x\∈D}{\min }(C(p_l,d)+\underset{q_s\∈N_1\left(p_l\right)\backslash q_l}{\mathrm{\Σ}}{M_{q_s{p}_l}}^{t-1}\left(d_x\right)+J(p_l,q_l,d,d_x)),$

P wherein _lBe arbitrary pixel in the left image, q _lBe pixel p _lAny neighbours territory pixel, D is candidate's disparity range of definition in the step 4.1, d is the arbitrary parallax value in the D, C (p _l, d) coupling cost, d for calculating among the step 4.2.2 _xBe the arbitrary parallax value in the disparity range D, J (p _l, q _l, d, d _x) the level and smooth cost of trying to achieve for step 7.1,

For t-1 iteration try to achieve from pixel q _sTo p _lThe parallax of propagating is d _xDegree of confidence message, during t=1

Be 0, d _xBe the arbitrary parallax value in the disparity range D, q _sBe pixel p _lSelectivity neighborhood N ₁(p _l) in, be different from pixel q _lArbitrary pixel, described selectivity neighborhood N ₁(p _l) be:

N ₁(p _l)＝{q _f|q _f∈N(p _l)，Con(q _f)≥Con(p _l)and?S(q _f)＝S(p _l)}，

N in the formula ₁(p _l) be pixel p _lThe territory of neighbours up and down, Con (q _f) and Con (p _l) be the fiduciary level coefficient of mark in the step 5.3, S (q _f) and S (p _l) be the pixel q that tries to achieve in the step 6 _fAnd p _lAffiliated cut zone label;

Step 7.3: calculate each pixel in the left image with respect to might parallax degree of confidence b (p _l, d): $b(p_l,d)=C(p_l,d)+\underset{p_s\∈N_1\left(p_l\right)}{\mathrm{\Σ}}{M}_{p_s{p}_l}^{50}\left(d\right),$ P wherein _lBe the arbitrary pixel in the left image, d is the arbitrary parallax value in the D, C (p _l, the coupling cost that d) obtains for step 4.2.2, Be the 50th iterative computation obtain from pixel p _sTo p _lThe parallax of propagating is the degree of confidence message of d, p _sBe N ₁(p _l) interior arbitrary pixel, N ₁(p _l) be the p of definition in the step 7.2 _lThe selectivity neighborhood;

Step 7.4: calculate anaglyph

The optimum parallax value d of confidence calculations (p according to each pixel _l):

$d\left(p_l\right)=\underset{d\∈D}{\arg \min }b(p_l,d),$

P wherein _lBe the arbitrary pixel in the left image, b (p _l, the degree of confidence that d) calculates for step 7.3, D is candidate's disparity range, d is the arbitrary parallax value in candidate's disparity range D;

Optimum parallax according to each pixel in the left image is set up final parallax as D ^Out: D ^Out(x, y)=d (p _Xy), wherein x and y are respectively anaglyph D ^OutThe horizontal ordinate of pixel and ordinate, p _XyBe that coordinate is (x, pixel y), d (p in the benchmark image _Xy) be p _XyOptimum parallax value;

Step 8: the three-dimensional information of reconstructed object thing

The camera interior and exterior parameter matrix A that obtains according to step 2 _L, A _R[R _Lt _L], [R _Rt _R], and the anaglyph D that obtains of step 7 ^Out, calculate the three-dimensional point cloud model of whole object through the space method of crossing.

Images