CN106228605A - A kind of Stereo matching three-dimensional rebuilding method based on dynamic programming - Google Patents

Abstract

The invention discloses a kind of Stereo matching three-dimensional rebuilding method based on dynamic programming, this system is made up of two video cameras, implementation step is: (1) is by adjusting the position of two video cameras, three-dimension measuring system is demarcated by the imaging plane the most parallel (2) making two video cameras: obtain the internal reference of two video cameras and outer ginseng, obtains the corresponding relation of the pixel coordinate on image and world coordinate system.(3) epipolar geom etry correction and image conversion (4) are utilized Stereo Matching Algorithm based on dynamic programming, draw disparity map.(5) parallax correction (6), according to camera calibration parameter and disparity map, obtains three-dimensional point cloud by space Convergence method.Disparity map precision of the present invention is high, real-time is high, and can accurate fast automatic reconstruction image three-dimensional point cloud.

Description

A kind of Stereo matching three-dimensional rebuilding method based on dynamic programming

Technical field

The invention belongs to technique of binocular stereoscopic vision field, the problem relating to process image based on Stereo matching.

Background technology

Technique of binocular stereoscopic vision is the method for three-dimensional measurement of a kind of passive type, and its implementation is flexible, to environmental requirement Low, man-machine interaction is friendly, is a kind of technology popular in three-dimensional reconstruction algorithm.It is double that binocular stereo vision is intended to simulating human The mechanism of the other scene three-dimensional information of outlook, obtains the two dimensional image of scenes from two angles, further according to set up between image Joining reconstruction of relations threedimensional model, mainly include camera calibration, image is to main process such as coupling, three-dimensional information reduction.Set up The process of two width image pixel point correspondences is exactly the process of Stereo matching, and it is the core of technique of binocular stereoscopic vision, main Wanting task is to first pass through binocular or many mesh images match obtains disparity map, is then obtained the scape of object by triangulation relation Deeply.

The main task of Stereo matching is to obtain smooth dense disparity map true to nature, and Stereo Matching Algorithm is broadly divided into local Algorithm and Global Algorithm, local algorithm utilizes the neighborhood information of pixel to mate, and computation complexity is relatively low, but matching precision The highest, easily produce mistake at low texture, parallax discontinuity zone especially.Flatness cost is added Matching power flow by Global Algorithm Calculating in, make coupling be converted into global optimum's process of energy function, mainly have figure to cut algorithm, belief propagation algorithm and dynamic State planning algorithm.Wherein dynamic programming algorithm computation complexity is minimum, fastest, but easily produces strip flaw problem, puts It is higher that reliability propagation algorithm and figure cut algorithmic match precision, calculated disparity map edge region and degree of depth discontinuity zone Effect preferable, comparatively speaking, it is time-consumingly long that figure cuts algorithm, and real-time performance is to be improved.

The shortcoming that existing three-dimensional reconstruction algorithm based on binocular stereo vision exists the following aspects:

(1) building the key that suitable neighborhood window is local algorithm, window is the least, then cannot comprise pixel to be matched Enough neighborhood informations, window is too big, then will comprise the neighborhood information without directive significance in the calculating of Matching power flow, these The generation of erroneous matching will be caused.

(2) global energy optimization is confined to one-dimensional by the dynamic programming algorithm that Global Algorithm complexity is relatively low Scan line in, lost the slickness constraint in other directions, can produce travers, and real-time can not get improving.Figure Cutting algorithm time-consuming relatively long, be difficult to meet the requirement of real-time of real scene shooting 3-dimensional reconstruction, belief propagation algorithm is indiscriminately Between neighborhood territory pixel, propagate confidence level, and disparity continuity may be unsatisfactory for about between parallax discontinuity zone neighborhood territory pixel Bundle, result causes reconstruction point cloud obscure boundary clear.

Owing to there is disadvantage mentioned above, existing three-dimensional reconstruction algorithm based on Stereo matching can not obtain in actual applications To gratifying result.Although it addition, in recent years, Stereo Matching Algorithm is studied by a large amount of scholars, more and more higher The method of precision is suggested.But a lot of algorithms are difficult to the double requirements reaching precision with real-time.

Summary of the invention

Goal of the invention: easily produce travers and the inadequate problem of real-time for conventional dynamic planning algorithm, this A kind of Stereo matching three-dimensional rebuilding method based on dynamic programming of bright offer, the present invention uses a kind of new self adaptation correlation function Calculate Matching power flow, merged colour and the gradient information of picture, then used matched filtering.Disparity map precision is improved, again Use hierarchical algorithm and control point method that real-time is improved, and can accurate fast automatic reconstruction image three-dimensional point Cloud.

Technical scheme: for achieving the above object, the technical solution used in the present invention is:

A kind of Stereo matching three-dimensional rebuilding method based on dynamic programming, comprises the following steps:

Step 1: Image Acquisition, uses two, the left and right video camera of binocular camera respectively to shoot piece image, wherein simultaneously The shooting of left video camera for left image, the shooting of right video camera for right image；

Two video cameras are demarcated, are set up camera review location of pixels and field by step 2: camera calibration respectively Relation between scape position, it is thus achieved that the Intrinsic Matrix A of left video camera_L, the Intrinsic Matrix A of right video camera_RWith left video camera Outer parameter matrix [R_L t_L], the outer parameter matrix [R of right video camera_R t_R]；

Step 3: polar curve is corrected by image, according to the inside and outside parameter matrix utilization pole of left and right two video cameras that step 2 obtains Line bearing calibration carries out polar curve correction and obtains run-in index binocular vision model the left and right image captured by step 1, makes coupling picture Element is to having identical vertical coordinate；Left image and right image after correction are designated as I respectively_lAnd I_r；

Step 4: utilize dynamic programming algorithm Stereo matching obtain disparity map, the left image corrected according to step 3 and Right image determines disparity range, with the left image I after correction_lOn the basis of image, with correction after right image I_rFor registrating image, Use adaptive weighting windowhood method that each pixel in benchmark image is calculated Matching power flow and to obtain initial left view poor Figure, then, with the right image I after correction_rOn the basis of image, with correction after left image I_lFor registrating image, in benchmark image Each pixel calculate Matching power flow obtain initial right disparity map；After having calculated Matching power flow, generate disparity space image； Use dynamic programming to ask for optimal path in disparity space image to be optimized, disparity map must be taken；

Step 5: parallax correction, it is judged that in benchmark image, whether pixel p is reliable point, and make the p obtained in step 4 Point final parallax value Dispartiy (p)=d_l(Npi), d_l(Npi) it is the parallax value of Npi, coordinate in image on the basis of Npi (x, The pixel of parallax unreliable pixel p eight neighborhood y), and then obtain final disparity map；

Step 6: the final disparity map D that the camera interior and exterior parameter matrix obtained according to step 2 and step 5 obtain, The three-dimensional point cloud model of whole object it is calculated by space Convergence method.

Preferred: in described step 4 before generating disparity space image, use local linear filtering that two width images are filtered Ripple.

Preferred: described local linear filtering method is the left image corrected step 3 and right image carries out height respectively This filtering, eliminates influence of noise.

Preferred: the method utilizing the Stereo matching of dynamic programming algorithm to obtain disparity map in described step 4 includes following Step:

Step 4.1: determine disparity range according to left and right two width image；

D=(d_min,d_max),

Wherein, d_minFor minimum parallax, d_min=0, d_maxFor maximum disparity, by labelling benchmark image and registration image it Between matched pixel point to trying to achieve；

Step 4.2: calculate Matching power flow, with the left image I after correction_lOn the basis of image, with correction after right image I_rFor Registration image, at the beginning of using adaptive weighting windowhood method each pixel in benchmark image to calculate Matching power flow and obtains Begin left disparity map, then, with the right image I after correction_rOn the basis of image, with correction after left image I_lFor registrating image, to base Each pixel in quasi-image calculates Matching power flow and obtains initial right disparity map:

C (p, d)=δ * C_TAD(p,d)+(1-δ)*C_Gradient(p,d)

$C_{TAD}(p,d)=min\left\{\underset{i=R,G,B}{\mathrm{\Σ}}\right|{R_i}^{\′}\left(p_d\right)-R_i\left(p\right),{\mathrm{\τ}}_{AD}\left|\right\}$

$\begin{array}{c}{C}_{Gradient}(p,d)=\min \left\{\underset{i=R,G,B}{\Σ}\right|{\▿}_x{R_i}^{\′}\left(p_d\right)-{\▿}_x{R}_i\left(p\right),{\mathrm{\τ}}_{{\▿}_x}\left|\right\}\\ +\min \left\{\underset{i=R,G,B}{\Σ}\right|{\▿}_y{R_i}^{\′}\left(p_d\right)-{\▿}_y{R}_i\left(p\right),{\mathrm{\τ}}_{{\▿}_y}\left|\right\}\end{array}$

Wherein, p is pixel, and d is the parallax value of this pixel, and R is coloured image, and parameter δ is used for balancing colour information C_TAD(p, d) with gradient information C_Gradient(p, d) between proportionate relationship；R_i'(p_d) it is the right image pixel at p point i passage Value, R_iP () is the left image pixel value at p point i passage,For Grad on x direction,For Grad, τ on y direction_ADTable Show coloured image space R, G, the interceptive value of channel B component；C_Gradient(p, d) represents along x and y direction, logical to R, G, B tri- Road carries out the calculating of phase threshold gradient；WithRepresent the interceptive value in x and y direction respectively；p_dFor at left image pixel Plus the pixel value of (in the rightest image) p point after parallax d in value；

Step 4.3: Matching power flow is filtered

Employing local linear filters, and for any one pixel p, filtered Matching power flow is:

${\hat{C}}_p\left(d\right)=\underset{q}{\Σ}{W}_{p,q}\×C_q\left(d\right);$

Wherein, W_p,qFor kernel function,

$W_{p,q}=\frac{1}{\left|w^2\right|}\underset{k:(p,q)\∈{\mathrm{\ω}}_k}{\Σ}(1+{\left(I_p-{\mathrm{\μ}}_k\right)}^T{\left({\mathrm{\Σ}}_k+\mathrm{\ϵ}U\right)}^{-1}(I_q-{\mathrm{\μ}}_k\left)\right)$

Above formula is the weights filter function of coloured image R, I_p, I_qAnd μ_kRepresent 3 × 1 vectors of colouring information, ω_kFor greatly The rectangular window of little 3 × 3, ∑_kRepresenting the covariance matrix of 3 × 3, U is the unit matrix of 3 × 3, and q is the neighborhood territory pixel of p

After having calculated Matching power flow and having been filtered by Matching power flow, generate disparity space image；

Step 4.4: use dynamic programming to ask for optimal path in disparity space image and be optimized, disparity map must be taken.

Preferred: by the matched pixel point between labelling benchmark image and registration image to trying to achieve in described step 4.1 The method of maximum disparity:

Randomly select ten pixels in benchmark image pl1, pl2, pl3 ..., pl10}, in registration image respectively Find with pl1, pl2, pl3 ..., pl10} has ten estimation matched pixel points of identical vertical coordinate and similar color information Pr1, pr2, pr3 ..., pr10}, then obtain ten groups estimation matched pixel to (pl1, pr1), (pl2, pr2), (pl3, Pr3) ..., (pl10, pr10) }, the difference of each group of matched pixel abscissa to calculating two pixels thoroughly deserved one group Parallax value d1, d2, d3 ..., and d10}, maximum disparity d_max=max{d1, d2 ..., d10}₊₅。

Preferred: described step 4.4 to use in disparity space image dynamic programming ask for optimal path and be optimized, The method taking disparity map: the Matching power flow in every scan line is added up by dynamic programming, finds smallest match from left to right Cost value path, thus obtain the parallax value of each pixel；On the basis of left images, image calculates left disparity map D respectively_l, right Disparity map D_r, use left and right conformance criteria, will meet | d_l(p)-d_r(q) | the point of≤1 is labeled as the reliable point of parallax, and makes Dispartiy (p)=(d_l(p)+d_r(q))/2；Otherwise it is labeled as the unreliable point of parallax and is designated as Dispartiy (p)=0；Wherein p Pixel in image on the basis of Dian, q point is the match point of p point, d in registration image_l(p)∈D_lFor the parallax value of pixel p, d_r (q)∈D_rFor the parallax of pixel q, Dispartiy (p) is the p final parallax value of point.

Preferred: the method obtaining final disparity map in described step 5:

By coordinate in benchmark image, (x, the pixel of parallax unreliable pixel p eight neighborhood y) is labeled as Npi, wherein (x_i,y_i) it is the image coordinate of Npi,The gray value of Npi is subtracted each other with p point gray value, obtains gray scale difference Value, and by gray scale difference value by order sequence from small to large；According to suitable to maximum pixel of the pixel minimum from gray scale difference value It is the reliable point of parallax that sequence judges whether that pixel Npi meets following three condition (1) Npi successively；(2)Np_i∈S_p, wherein S_pWindow ranges for pixel place；(3)|I_l(x_i,y_i)-I_r(x_i+d_l(Npi),y_i) |≤s, wherein I_l(), I_r() represents The gray value of pixel, d in benchmark image and registration image_l(Npi)∈D_lFor the parallax value of Npi, s is the threshold value set；

If there is Npi to meet three conditions above, p point being labeled as reliable point, and makes Dispartiy (p)=d_l (Npi)；Otherwise condition (3) is replaced with | I_l(x,y)-I_l(x+m, y+n) |≤s recalculates condition (1) (2) (3), if deposited Meet condition at Npi p is then labeled as parallax reliably to put and make parallax Dispartiy (p)=d_l(Npi)；Wherein m, n ∈ (-1, 0,1) it is, 0 during m, n difference；Final disparity map is obtained through this step.

Beneficial effect: a kind of based on dynamic programming the Stereo matching three-dimensional rebuilding method that the present invention provides, compares existing Technology, has the advantages that

Propose a kind of self adaptation similarity measure function, introduce colour information and the gradient information of RGB image, improve Disparity map precision.The complexity of auto-adaptive function makes efficiency of algorithm reduce, the present invention use again control point hierarchical algorithm with Improve arithmetic speed.Through comparing, the present invention all reaches requirement in precision with real-time.

Accompanying drawing explanation

Fig. 1 is the whole process flow diagram flow chart of the present invention.

Fig. 2 is system model and principle schematic.

Fig. 3 is that polar curve polar curve corrects schematic diagram.

Fig. 4 is disparity space image schematic diagram.

Fig. 5 is searching route schematic diagram.

Fig. 6 is calculated the 3 d space coordinate schematic diagram of object point on image by matching relationship and nominal data.

Detailed description of the invention

Below in conjunction with the accompanying drawings and specific embodiment, it is further elucidated with the present invention, it should be understood that these examples are merely to illustrate this Invention rather than limit the scope of the present invention, after having read the present invention, various to the present invention of those skilled in the art The amendment of the equivalent form of value all falls within the application claims limited range.

With reference to the accompanying drawings, specific embodiments of the present invention are done more detailed description.Programming realization tools selection Visual Studio2013, in selection Middlebury java standard library, image is as coupling image.

Fig. 1 is the entire flow of the present invention.

Fig. 2 is system model and the principle schematic of the present invention.Use two CCD the most each from two different angles respectively Shoot a width building image, O_L、O_RIt is respectively the photocentre of two video cameras, I_L、I_RThe imaging being respectively two video cameras is put down Face, P is a dimensional target point on building to be reconstructed, P_L, P_RThe institute in two video camera imaging planes respectively for object point P The picture point become.This by the same space object point on different cameras imaging plane formed picture point be a pair match point.Appoint and take Wherein image on the basis of a width, another width is registration image, on the basis of each pixel in image search in alignment image The process of corresponding match point is referred to as Stereo matching.After obtaining the matching relationship of pixel, according to system model, in conjunction with demarcating The camera interior and exterior parameter arrived, carries out reverse operation, so that it may obtains the 3 d space coordinate of corresponding object point, thus realizes image Three-dimensionalreconstruction.

It is that polar curve is corrected schematic diagram as shown in Figure 3.The inside and outside parameter obtained according to calibration process in step 2, uses ginseng Examine document " A compact algorithm for rectification of stereo pairs.Machine Vision And Applications " polar curve of proposition in (Fusiello A, Trucco E, Verri A.2000,12 (1): 16-22) Captured left images is carried out correcting, polar curve if the pixel coordinate after Bian Huan in image corresponds to former by bearing calibration Time in non-integer coordinates in beginning image, then carry out by gray scale bilinear interpolation, finally obtain run-in index binocular vision model, Make matched pixel to being in same scan line, reduce coupling space complexity, corrected after image undistorted, correction Accuracy rate is high, and error is less than a pixel.

Its purpose of the present invention, for generating a secondary final parallax, is carried out scene by the camera parameters of disparity map and acquisition Three-dimensional reconstruction.First gather two width colour real scene shooting images, carry out camera calibration, carry out polar curve is corrected according to nominal data Convert with image, then ask for the disparity space image of image pair, use a kind of self adaptation similarity to survey in disparity space image Degree function calculates Matching power flow and carries out cost filtering.Dynamic programming algorithm is used to find optimal path in disparity space image again Ask for parallax, after parallax correction, finally utilize nominal data and matching result rebuild three-dimensional point cloud and show.

The present invention to be embodied as step as follows:

Step 1: Image Acquisition

Use binocular camera to obtain image, first adjust binocular camera and make its optical axis substantially parallel and make left and right pitch-angle Degree is in suitable position, then respectively shoots piece image simultaneously, the most left lens shooting for left image, right lens shooting For right image；

Step 2: camera calibration

Respectively two video cameras are demarcated, set up the relation between camera review location of pixels and scene location, Obtain the Intrinsic Matrix A of the video camera on the left side_L, the Intrinsic Matrix A of video camera on the right_ROuter ginseng with the video camera on the left side Matrix number [R_L t_L], the outer parameter matrix [R of the video camera on the right_R t_R]；

Step 3: polar curve is corrected by image

The camera interior and exterior parameter obtained according to step 2 uses method for correcting polar line that captured left images is carried out pole Line correction obtain run-in index binocular vision model, make matched pixel to having identical vertical coordinate, even if matched pixel is to being in With in scan line.Left image and right image after correction are designated as I respectively_lAnd I_r；

Step 4: utilize the Stereo matching of dynamic programming algorithm to obtain disparity map

Step 4.1 carries out gaussian filtering to two width images, eliminates influence of noise and improves picture quality；

Step 4.2: the initial matching cost of two width images about calculating, sets up disparity space image (DSI)

Step 4.2.1: determine disparity range

D=(d_min,d_max),

Wherein d_minFor minimum parallax, d_min=0, d_maxFor maximum disparity, by between labelling benchmark image and registration image Matched pixel point to trying to achieve:

Randomly select ten pixels in benchmark image pl1, pl2, pl3 ..., pl10}, in registration image respectively Find with pl1, pl2, pl3 ..., pl10} has ten estimation matched pixel points of identical vertical coordinate and similar color information Pr1, pr2, pr3 ..., pr10}, then obtain ten groups estimation matched pixel to (pl1, pr1), (pl2, pr2), (pl3, Pr3) ..., (pl10, pr10) }, the difference of each group of matched pixel abscissa to calculating two pixels thoroughly deserved one group Parallax value d1, d2, d3 ..., and d10}, maximum disparity d_max=max{d1, d2 ..., d10}₊₅；

Step 4.2.2: calculate Matching power flow

With the left image I after correction_lOn the basis of image, with correction after right image I_rFor registration image, use self adaptation power Weight windowhood method calculates Matching power flow to each pixel in benchmark image and obtains initial left disparity map, then, with school Right image I after just_rOn the basis of image, with correction after left image I_lFor registrating image, to each pixel in benchmark image Point calculates Matching power flow and obtains initial right disparity map:

C (p, d)=δ * C_TAD(p,d)+(1-δ)*C_Gradient(p,d)

$C_{TAD}(p,d)=min\left\{\underset{i=R,G,B}{\Σ}\right|{R_i}^{\′}\left(p_d\right)-R_i\left(p\right),{\mathrm{\τ}}_{AD}\left|\right\}$

$C_{Gradient}(p,d)=\min \left\{\underset{i=R,G,B}{\Σ}\right|{\▿}_x{R_i}^{\′}\left(p_d\right)-{\▿}_x{R}_i\left(p\right),{\mathrm{\τ}}_{{\▿}_x}\left|\right\}+\min \left\{\underset{i=R,G,B}{\Σ}\right|{\▿}_y{R_i}^{\′}\left(p_d\right)-{\▿}_y{R}_i\left(p\right),{\mathrm{\τ}}_{{\▿}_y}\left|\right\}$

Wherein, p is pixel, and d is the parallax value of this pixel, and R is coloured image, and parameter δ is used for balancing colour information C_TAD(p, d) with gradient information C_Gradient(p, d) between proportionate relationship；R_i'(p_d) it is the right image pixel at p point i passage Value, R_iP () is the left image pixel value at p point i passage,For Grad on x direction,For Grad, τ on y direction_ADTable Show coloured image space R, G, the interceptive value of channel B component；C_Gradient(p, d) represents along x and y direction, logical to R, G, B tri- Road carries out the calculating of phase threshold gradient；WithRepresent the interceptive value in x and y direction respectively；p_dFor at left image pixel Plus the pixel value of (in the rightest image) p point after parallax d in value；

Step 4.2.3: Matching power flow is filtered:

Employing local linear filters, and for any one pixel p, filtered Matching power flow is:

${\hat{C}}_p\left(d\right)=\underset{q}{\Σ}{W}_{p,q}\×C_q\left(d\right)$

Wherein, W_p,qFor kernel function,

$W_{p,q}=\frac{1}{\left|w^2\right|}\underset{k:(p,q)\∈{\mathrm{\ω}}_k}{\Σ}(1+{\left(I_p-{\mathrm{\μ}}_k\right)}^T{\left({\mathrm{\Σ}}_k+\mathrm{\ϵ}U\right)}^{-1}(I_q-{\mathrm{\μ}}_k\left)\right)$

Above formula is the weights filter function of coloured image R, I_p, I_qAnd μ_kRepresent 3 × 1 vectors of colouring information, ω_kFor greatly The rectangular window of little 3 × 3, ∑_kRepresenting the covariance matrix of 3 × 3, U is the unit matrix of 3 × 3, and q is the neighborhood territory pixel of p.Meter After having calculated Matching power flow and having been filtered by Matching power flow, disparity space image can be generated.

Step 4.3: use dynamic programming to ask for optimal path in disparity space image and be optimized, disparity map must be taken.Dynamic Matching power flow in every scan line is added up by state planning, finds smallest match cost value path from left to right, thus obtains Take the parallax value of each pixel.On the basis of left images, image calculates disparity map D respectively_l, D_r, use left and right conformance criteria, To meet | d_l(p)-d_r(q) | the point of≤1 is labeled as the reliable point of parallax, and makes Dispartiy (p)=(d_l(p)+d_r(q))/2； Otherwise it is labeled as the unreliable point of parallax and is designated as Dispartiy (p)=0；

Wherein pixel in image on the basis of p point, q point is the match point of p point, d in registration image_l(p)∈D_lFor pixel The parallax value of p, d_r(q)∈D_rFor the parallax of pixel q, Dispartiy (p) is the p final parallax value of point；

Step 5: parallax correction

Step 5.1: fill the unreliable point of parallax

By coordinate in benchmark image, (x, the pixel of parallax unreliable pixel p eight neighborhood y) is labeled as Npi, wherein (x_i,y_i) it is the image coordinate of Npi,The gray value of Npi is subtracted each other with p point gray value, obtains gray scale difference Value, and by gray scale difference value by order sequence from small to large；According to suitable to maximum pixel of the pixel minimum from gray scale difference value It is the reliable point of parallax that sequence judges whether that pixel Npi meets following three condition (1) Npi successively；(2)Np_i∈S_p, wherein S_pWindow ranges for pixel place；(3)|I_l(x_i,y_i)-I_r(x_i+d_l(Npi),y_i) |≤s, wherein I_l(), I_r() represents The gray value of pixel, d in benchmark image and registration image_l(Npi)∈D_lFor the parallax value of Npi, s is the threshold value set；

If there is Npi to meet three conditions above, p point being labeled as reliable point, and makes Dispartiy (p)=d_l (Npi)；Otherwise condition (3) is replaced with | I_l(x,y)-I_l(x+m, y+n) |≤s recalculates condition (1) (2) (3), if deposited Meet condition at Npi p is then labeled as parallax reliably to put and make parallax Dispartiy (p)=d_l(Npi)；Wherein m, n ∈ (-1, 0,1) it is, 0 during m, n difference；Final disparity map is obtained through this step；

Step 6: according to camera calibration parameter and disparity map, obtain three-dimensional point cloud model by space Convergence method

The camera interior and exterior parameter matrix A obtained according to step 2_L、A_R[R_L t_L]、[R_R t_R], and step 5 obtains Disparity map D, is calculated the three-dimensional point cloud model of whole object by space Convergence method.

Hierarchical mode is added conventional dynamic planing method by the present invention, provides with low pixel level for high pixel level and controls Point set, and use a kind of self adaptation relevance measure function in Matching power flow calculates, in addition Matching power flow filtering, improve algorithm Degree of accuracy and real-time also obtain high accuracy disparity map, then utilized space Convergence method to enter by acquired disparity map and camera parameter Row three-dimensional reconstruction.

The above is only the preferred embodiment of the present invention, it should be pointed out that: for the ordinary skill people of the art For Yuan, under the premise without departing from the principles of the invention, it is also possible to make some improvements and modifications, these improvements and modifications also should It is considered as protection scope of the present invention.

Claims (7)

1. a Stereo matching three-dimensional rebuilding method based on dynamic programming, it is characterised in that comprise the following steps:

Step 1: Image Acquisition, uses two, the left and right video camera of binocular camera respectively to shoot piece image simultaneously, and wherein a left side is taken the photograph Camera shooting for left image, the shooting of right video camera for right image；

Two video cameras are demarcated, are set up camera review location of pixels and scene bit by step 2: camera calibration respectively Relation between putting, it is thus achieved that the Intrinsic Matrix A of left video camera_L, the Intrinsic Matrix A of right video camera_ROuter ginseng with left video camera Matrix number [R_L t_L], the outer parameter matrix [R of right video camera_R t_R]；

Step 3: polar curve is corrected by image, according to the inside and outside parameter matrix utilization polar curve school of left and right two video cameras that step 2 obtains Correction method carries out polar curve correction and obtains run-in index binocular vision model the left and right image captured by step 1, makes matched pixel pair There is identical vertical coordinate；Left image and right image after correction are designated as I respectively_lAnd I_r；

Step 4: utilize the Stereo matching of dynamic programming algorithm to obtain disparity map, the left image corrected according to step 3 and right figure As determining disparity range, with the left image I after correction_lOn the basis of image, with correction after right image I_rFor registration image, use Adaptive weighting windowhood method calculates Matching power flow to each pixel in benchmark image and obtains initial left disparity map, so After, with the right image I after correction_rOn the basis of image, with correction after left image I_lFor registration image, every in benchmark image One pixel calculates Matching power flow and obtains initial right disparity map；After having calculated Matching power flow, generate disparity space image；Regarding Difference space figure uses dynamic programming ask for optimal path to be optimized, disparity map must be taken；

Step 5: parallax correction, it is judged that in benchmark image, whether pixel p is reliable point, and make the p point that obtains in step 4 Whole parallax value Dispartiy (p)=d_l(Npi), d_l(Npi) it is the parallax value of Npi, coordinate in image on the basis of Npi (x, y) The pixel of parallax unreliable pixel p eight neighborhood, and then obtain final disparity map；

Step 6: the final disparity map D that the camera interior and exterior parameter matrix obtained according to step 2 and step 5 obtain, passes through Space Convergence method is calculated the three-dimensional point cloud model of whole object.

Stereo matching three-dimensional rebuilding method based on dynamic programming the most according to claim 1, it is characterised in that: described step In rapid 4 before generating disparity space image, use local linear filtering that two width images are filtered.

Stereo matching three-dimensional rebuilding method based on dynamic programming the most according to claim 2, it is characterised in that: described office Portion's linear filter method is the left image corrected step 3 and right image carries out gaussian filtering respectively, eliminates influence of noise.

Stereo matching three-dimensional rebuilding method based on dynamic programming the most according to claim 3, it is characterised in that: described step The Stereo matching utilizing dynamic programming algorithm in rapid 4 obtains the method for disparity map and comprises the following steps:

Step 4.1: determine disparity range according to left and right two width image；

D=(d_min,d_max),

Wherein, d_minFor minimum parallax, d_min=0, d_maxFor maximum disparity, by between labelling benchmark image and registration image Matched pixel point is to trying to achieve；

Step 4.2: calculate Matching power flow, with the left image I after correction_lOn the basis of image, with correction after right image I_rFor registration Image, uses adaptive weighting windowhood method each pixel in benchmark image is calculated Matching power flow and obtains an initial left side Disparity map, then, with the right image I after correction_rOn the basis of image, with correction after left image I_lFor registrating image, to reference map Each pixel in Xiang calculates Matching power flow and obtains initial right disparity map:

C (p, d)=δ * C_TAD(p,d)+(1-δ)*C_Gradient(p,d)

$C_{TAD}(p,d)=min\left\{\underset{i=R,G,B}{\Σ}\right|{R_i}^{\′}\left(p_d\right)-R_i\left(p\right),{\mathrm{\τ}}_{AD}\left|\right\}$

$\begin{array}{c}{C}_{Gradient}(p,d)=\min \left\{\underset{i=R,G,B}{\Σ}\right|{\▿}_x{R_i}^{\′}\left(p_d\right)-{\▿}_x{R}_i\left(p\right),{\mathrm{\τ}}_{{\▿}_x}\left|\right\}\\ +\min \left\{\underset{i=R,G,B}{\Σ}\right|{\▿}_y{R_i}^{\′}\left(p_d\right)-{\▿}_y{R}_i\left(p\right),{\mathrm{\τ}}_{{\▿}_y}\left|\right\}\end{array}$

Wherein, p is pixel, and d is the parallax value of this pixel, and R is coloured image, and parameter δ is used for balancing colour information C_TAD (p, d) with gradient information C_Gradient(p, d) between proportionate relationship；R_i'(p_d) it is the right image pixel value at p point i passage, R_i P () is the left image pixel value at p point i passage,For Grad on x direction,For Grad, τ on y direction_ADRepresent coloured silk Color image space R, G, the interceptive value of channel B component；C_Gradient(p d) represents along x and y direction, to R, tri-passages of G, B enter The calculating of row order section threshold gradient；WithRepresent the interceptive value in x and y direction respectively；p_dFor on left image pixel value Plus the pixel value of p point (in the rightest image) after parallax d；

Step 4.3: Matching power flow is filtered

Employing local linear filters, and for any one pixel p, filtered Matching power flow is:

${\hat{C}}_p\left(d\right)=\underset{q}{\Σ}{W}_{p,q}\×C_q\left(d\right);$

Wherein, W_p,qFor kernel function,

$W_{p,q}=\frac{1}{\left|w^2\right|}\underset{k:(p,q)\∈{\mathrm{\ω}}_k}{\Σ}(1+{\left(I_p-{\mathrm{\μ}}_k\right)}^T{\left({\mathrm{\Σ}}_k+\mathrm{\ϵ}U\right)}^{-1}(-I_q-{\mathrm{\μ}}_k\left)\right)$

Above formula is the weights filter function of coloured image R, I_p, I_qAnd μ_kRepresent 3 × 1 vectors of colouring information, ω_kSized by 3 × 3 Rectangular window, ∑_kRepresenting the covariance matrix of 3 × 3, U is the unit matrix of 3 × 3, and q is the neighborhood territory pixel of p；

After having calculated Matching power flow and having been filtered by Matching power flow, generate disparity space image；

Step 4.4: use dynamic programming to ask for optimal path in disparity space image and be optimized, disparity map must be taken.

Stereo matching three-dimensional rebuilding method based on dynamic programming the most according to claim 4, it is characterised in that: described step By the matched pixel point method to trying to achieve maximum disparity between labelling benchmark image and registration image in rapid 4.1:

Randomly select ten pixels in benchmark image pl1, pl2, pl3 ..., pl10}, find respectively in registration image With pl1, pl2, pl3 ..., pl10} have identical vertical coordinate and similar color information ten estimation matched pixel points pr1, Pr2, pr3 ..., pr10}, then obtain ten groups estimation matched pixel to (pl1, pr1), (pl2, pr2), (pl3, pr3) ..., (pl10, pr10) }, the difference of each group of matched pixel abscissa to calculating two pixels thoroughly deserved one group of parallax value D1, d2, d3 ..., and d10}, maximum disparity d_max=max{d1, d2 ..., d10}+5.

Stereo matching three-dimensional rebuilding method based on dynamic programming the most according to claim 5, it is characterised in that: described step Use dynamic programming to ask for optimal path in rapid 4.4 in disparity space image to be optimized, the method that disparity map must be taken: dynamically advise Draw and the Matching power flow in every scan line is added up, find smallest match cost value path from left to right, thus obtain every The parallax value of one pixel；On the basis of left images, image calculates left disparity map D respectively_l, right disparity map D_r, consistent about utilization Property criterion, will meet | d_l(p)-d_r(q) | the point of≤1 is labeled as the reliable point of parallax, and makes Dispartiy (p)=(d_l(p)+d_r (q))/2；Otherwise it is labeled as the unreliable point of parallax and is designated as Dispartiy (p)=0；Wherein pixel, q in image on the basis of p point Point is the match point of p point, d in registration image_l(p)∈D_lFor the parallax value of pixel p, d_r(q)∈D_rFor the parallax of pixel q, Dispartiy (p) is the p final parallax value of point.

Stereo matching three-dimensional rebuilding method based on dynamic programming the most according to claim 2, it is characterised in that: described step The method obtaining final disparity map in rapid 5:

By coordinate in benchmark image, (x, the pixel of parallax unreliable pixel p eight neighborhood y) is labeled as Npi, wherein (x_i, y_i) it is the image coordinate of Npi,The gray value of Npi is subtracted each other with p point gray value, obtains gray scale difference value, and By gray scale difference value by order sequence from small to large；Depend on to the order of maximum pixel according to the pixel minimum from gray scale difference value It is secondary that to judge whether that pixel Npi meets following three condition (1) Npi be the reliable point of parallax；(2)Np_i∈S_p, wherein S_pFor picture The window ranges at element place；(3)|I_l(x_i,y_i)-I_r(x_i+d_l(Npi),y_i) |≤s, wherein I_l(), I_r() represents reference map The gray value of pixel, d in picture and registration image_l(Npi)∈D_lFor the parallax value of Npi, s is the threshold value set；

If there is Npi to meet three conditions above, p point being labeled as reliable point, and makes Dispartiy (p)=d_l(Npi)；No Then condition (3) is replaced with | I_l(x,y)-I_l(x+m, y+n) |≤s recalculates condition (1) (2) (3), meets if there is Npi P is then labeled as parallax and reliably puts and make parallax Dispartiy (p)=d by condition_l(Npi)；Wherein m, n ∈ (-1,0,1), m, n It is asynchronously 0；Final disparity map is obtained through this step.