CN104867133A - Quick stepped stereo matching method - Google Patents

Abstract

A quick stepped stereo matching method comprises three steps of gray scale transformation based on a local texture characteristic, parallax restriction based on colorful image segmentation, and matching cost calculation based on a fixed window. Firstly, gray scale transformation is performed a image according to the local texture characteristic of the image for highlighting the structure characteristic and the gray scale characteristic of the image. Then the image is segmented according to a mean-shift algorithm, and the matching cost of a segmented region in a manner that the segmented region with a random size and random shape is used as a supporting window, and forming a parallax restriction based on image segmentation. Finally, stereo matching cost calculation based on a fixed window is performed on the image, thereby obtaining an initial parallax graph. The quick stepped stereo matching method has advantages of improving matching precision and matching efficiency, reducing calculation amount, reducing a parallax searching range, etc.

Description

One is substep solid matching method fast

Technical field

The present invention relates to a kind of computer vision and digital image processing field, especially one substep solid matching method fast.

Background technology

At present, Stereo matching (Stereo Matching) is the basis of theory on computer vision and application, is again a more scabrous problem simultaneously.The target of Stereo matching is from different visual point image, find the corresponding point of coupling, and it is widely used in the numerous areas such as remote sensing images, medical image, three-dimensionalreconstruction, robot vision and image measurement.There is a lot of Chinese scholars to conduct in-depth research this field all the time.Wherein, Szeliski and Scharsten is studied some representational Stereo Matching Algorithm and evaluates, and is summed up by all algorithms and is divided into four large steps, and existing Stereo Matching Algorithm is divided into local algorithm and Global Algorithm two class.Generally speaking, Global Algorithm precision is higher, but calculates loaded down with trivial details, and setting parameter is complicated; Local algorithm efficiency is high and be easy to realize, but there is the problem of the size and shape of how self-adaptation determination support window always, causes it to be difficult to reach higher precision.Chinese scholars proposes much representative local matching algorithm for this problem, and main summary is divided into following three classes:

The first kind concentrates on selects optimum window as support window in multiple windows given in advance.This algorithm improves the precision of images match to a certain extent, prospect is inhibit to amplify phenomenon (foregroundfattening effect), but because the selection of support window size and shape is defined, lack dirigibility, so be difficult to adapt to changeable picture structure, error hiding rate is still higher, and parallax edge is also clear not.

Equations of The Second Kind concentrates on and is weighted pixel in the support window determining size and shape, or self-adaptation chooses the size and shape of support window.Although this algorithm makes the ambiguousness of mating greatly reduce, the parallax result obtained can match in excellence or beauty with global optimization result, and due to calculation of complex, computing overhead is large, cannot embody the high efficiency of local algorithm.

3rd class concentrates on the similarity measure improving coupling.Zabih proposes the algorithm that a kind of non-parametric transformations carries out Stereo matching.Afterwards, a lot of scholar improved the Stereo matching based on non-parametric transformations, and achieved good effect.Although non-parametric transformations has good capacity switching signal, certain robustness is had to noise and amplitude distortion, and the impact of illumination on pixel grey scale can be balanced, but just simply non-parametric transformations is carried out to gray-scale value originally, do not utilize the cross-correlation information between pixel.

In sum, local matching algorithm has the shortcoming being difficult to take into account matching precision and matching efficiency, so while guarantee Stereo matching precision, the speed improving coupling is most important.

Summary of the invention

The object of the invention is to provide a kind of fast and fractional solid matching method based on local grain characteristic and color images can taking into account matching precision and matching efficiency.

For achieving the above object, have employed following technical scheme, matching process of the present invention comprises the following steps:

(1) input the coloured image of the same scene that two width are taken under different angles, will wherein piece image as with reference to image A ₁, another piece image is as target image B ₁, suppose that two width images are corrected through polar curve all;

(2) respectively to reference picture A ₁with target image B ₁carry out the greyscale transformation based on local grain characteristic, obtain gray level image to A ₂and B ₂;

(3) utilize mean shift algorithm to reference picture A ₁carry out color images, adopt zero-mean normalized crosscorrelation metric function, using the cut zone of arbitrary size and shape as support window to gray level image to A ₂and B ₂carry out Stereo matching, the Matching power flow value in computed segmentation region, Matching power flow value according to cut zone is classified to each color segmentation region, when cut zone Matching power flow is more than or equal to 0.5, cut zone is categorized as reliable matching region, be labeled as 1, when cut zone Matching power flow is less than 0.5, cut zone is categorized as unreliable matching area, is labeled as 0, estimate the disparity range of pixel in regional according to classification results, form the disparity constraint based on color images;

(4) associated class is adopted to estimate normalized crosscorrelation metric function (Normalized cross-correlation respectively, NCC) and distance-like estimate the absolute value of gray scale difference and metric function (Sum of AbsoluteDifference, SAD) to gray level image to A ₂and B ₂carry out the Stereo matching based on stationary window, and utilize the disparity constraint based on color images to limit disparity search scope, obtain reference picture A ₁disparity map.

Compared with prior art, tool of the present invention has the following advantages:

1, based on local grain characteristic greyscale transformation, make full use of the local grain characteristic of image, improve matching precision, the matching precision of especially low texture region.

2, based on the disparity constraint of color images, in the process of carrying out Stereo matching, greatly reduce disparity search scope, reduce calculated amount, improve matching efficiency, also improve the matching precision of occlusion area simultaneously.

3, when not carrying out parallax optimization and self-adapting window selects, no matter be select associated class to estimate NCC, or chosen distance class estimates SAD, all achieves good disparity map, has metric function accommodation widely.

Accompanying drawing explanation

Fig. 1 is the process flow diagram of the inventive method.

Fig. 2 is a) reference picture A in embodiment one ₁.

Fig. 2 b) be target image B in embodiment one ₁.

Fig. 3 a) be in embodiment one to reference picture A ₁carry out the gray level image A based on obtaining after the greyscale transformation of local grain characteristic ₂.

Fig. 3 b) be in embodiment one to reference picture A ₁the gray level image A obtained after carrying out ordinary gamma conversion ₃.

Fig. 3 c) be in embodiment one to target image B ₁carry out the gray level image B based on obtaining after the greyscale transformation of local grain characteristic ₂.

Fig. 3 d) be in embodiment one to target image B ₁the gray level image B obtained after carrying out ordinary gamma conversion ₃.

Fig. 4 be in embodiment one to reference picture A ₁the image obtained after carrying out color images.

Fig. 5 is a) the initial parallax figure that employing distance-like in embodiment one is estimated SAD and obtained.

Fig. 5 b) be the initial parallax figure that employing associated class in embodiment one is estimated NCC and obtained.

Embodiment

Below in conjunction with accompanying drawing, the present invention will be further described:

Process flow diagram of the present invention as shown in Figure 1, matching process of the present invention comprises the following steps:

(1) as shown in Figure 2, obtain the coloured image of the same scene that two width are taken under different angles and Input matching system, Fig. 2 is a) as with reference to image A ₁, Fig. 2 b) and as target image B ₁, two width images are corrected through polar curve all;

(2) respectively to reference picture A ₁with target image B ₁carry out the greyscale transformation based on local grain characteristic, obtain gray level image to A ₂and B ₂, so that while carrying out greyscale transformation, the local grain characteristic maximally utilising image carrys out architectural feature and the gray feature of outstanding image; As shown in Figure 3, Fig. 3 is a) reference picture A ₁through based on the gray level image A after the greyscale transformation of local grain characteristic ₂, Fig. 3 b) and be target image B ₁through based on the gray level image B after the greyscale transformation of local grain characteristic ₂.First coloured image is changed into gray level image A ₃and B ₃, then carry out the greyscale transformation based on local grain characteristic on this basis, the concrete computation process based on the greyscale transformation of local grain characteristic is as follows:

First, centered by object pixel p, get the mapping window W that a size is m × n, calculate the local grain contrast value Δ m of all pixels in this window, gray-scale value I (p) of center pixel is changed into I (p)+Δ m; Wherein, m is the length of mapping window, and n is the wide of mapping window;

Then, the size of the center pixel gray-scale value after comparing each neighborhood territory pixel and changing, sues for peace all pixel grey scales being greater than center pixel, by summed result assignment to center pixel.

Local grain contrast value Δ m represents the texture features between pixel, and its computing method are:

Add up be designated as sum being greater than I (the p)+gray-scale value of Δ m in window W ₁, number is n ₁; Be less than I (the p)+gray-scale value of Δ m to add up and be designated as sum ₂, number is n ₂, then have:

(3) utilize mean shift algorithm to reference picture A ₁carry out color images, segmentation result as shown in Figure 4; Then, adopt zero-mean normalized crosscorrelation metric function, using the cut zone of arbitrary size and shape as support window to gray level image to A ₂and B ₂carry out Stereo matching, the Matching power flow value in computed segmentation region, the disparity search scope of pixel in each color segmentation region is judged according to the scope of the Matching power flow value of cut zone, when cut zone Matching power flow is more than or equal to 0.5, cut zone is categorized as reliable matching region, be labeled as 1, when cut zone Matching power flow is less than 0.5, cut zone is categorized as unreliable matching area, be labeled as 0, estimate the disparity range of pixel in regional according to classification results, form the disparity constraint based on color images, concrete computation process is as follows:

Known A ₂and B ₂be respectively reference picture and target image through based on the image after the greyscale transformation of local grain characteristic.Define any pair matching area (s _p, s _q) when suppose parallax be d cut zone Matching power flow be E (s _p, s _q, d):

E(s _p,s _q,d)＝C _zncc(p,q,d)

$C_{\mathrm{zncc}}(p,q,d)=\frac{\underset{p\∈s_p,q\∈s_q}{\mathrm{\Σ}}(p-\stackrel{\‾}{p})\·(q-\stackrel{\‾}{q})}{\sqrt{\underset{p\∈s_p}{\mathrm{\Σ}}{(p-\stackrel{\‾}{p})}^2\underset{q\∈s_q}{\mathrm{\Σ}}{(q-\stackrel{\‾}{q})}^2}}$

Wherein, s _pit is reference picture any one cut zone after color segmentation; s _qregion corresponding with it in target image, i.e. s _p=s _q-d; D is the parallax value of hypothesis; P is cut zone s _pin any one pixel; Q is region s _qin a corresponding with it pixel, i.e. p ∈ s _p, q ∈ s _qand p=q-d; C _zncc(p, q, d) is cut zone s _pand corresponding region s _qsimilarity, adopt zero-mean normalized crosscorrelation metric function (Zero mean normalized cross-correlation, ZNCC) to estimate this similarity, and it can be used as the Matching power flow E (s of cut zone _p, s _q, d); cut zone s _pin the average of all grey scale pixel values; cut zone s _qin the average of all grey scale pixel values;

Utilize local optimization methods (Winner-Take-All) to maximize cut zone Matching power flow and obtain parallax value d _s, as the estimating disparity of pixels all in cut zone, that is:

d _s＝max E(s _p,s _q,d)；d∈[d _min,d _max]

In formula, d _minminimum parallax value, d _maxit is maximum disparity value;

Cut zone Matching power flow calculates and only need carry out color segmentation to reference picture.Color images is supposed based on smooth surface, and in the region that namely pixel color values is similar, the depth value of pixel is general also identical, and what these regions were often described is plane scene, and parallax saltus step place is the edge of cut zone.According to this hypothesis, all pixels in same cut zone have identical Matching power flow E (s _p, s _q, d), namely have identical best parallax _ds.

Cut zone Matching power flow E (s _p, s _q, scope d) is between-1 to 1, represents the similarity degree of two cut zone.Cut zone is more similar, and the Matching power flow of cut zone is higher, and namely matching degree is higher.So, with cut zone Matching power flow for foundation, matching result is classified, and give certain constraint criterion formation disparity constraint to each classification results, limit the disparity search scope of next step coupling.

Classify to the result of cut zone Matching power flow, principle of classification is as cut zone Matching power flow E (s _p, s _q, when d)>=0.5, cut zone s _pbe classified as reliable matching region, be designated as 1; As cut zone Matching power flow E (s _p, s _q, d) during < 0.5, cut zone s _pbe classified as unreliable matching area, be designated as 0; Cut zone is should s herein _p, s _qit is region corresponding with it;

$s_p=\left\{\begin{array}{c}1,E(s_p,s_q,d)\&GreaterEqual;0.5\\ 0,E(s_p,s_q,d)<0.5\end{array}\right.$

Estimate that the disparity range of pixel in regional forms the disparity constraint of substep coupling according to classification results, constraint criterion is defined as:

d(s _p)∈[down,up]

In formula, down is the minimum value of carrying out disparity range after disparity constraint, and up is the maximal value of carrying out disparity range after disparity constraint;

Work as s _pwhen=1,

$\left\{\begin{array}{c}\mathrm{down}\&Element;\max (0,d_s-2)\\ \mathrm{up}\&Element;\min (d_{\max },d_s+2)\end{array}\right.$

Work as s _pwhen=0,

$\left\{\begin{array}{c}\mathrm{down}\&Element;\max (0,d_s-0.5\&times;d_{\max })\\ \mathrm{up}\&Element;\min (d_{\max },d_s+0.5\&times;d_{\max })\end{array}\right.$

Wherein d (s _p) be cut zone s _pin all pixel parallax spans.

(4) associated class is adopted to estimate normalized crosscorrelation metric function (Normalized cross-correlation respectively, NCC) and distance-like estimate the absolute value of gray scale difference and metric function (Sum of AbsoluteDifference, SAD) to gray level image to A ₂and B ₂carry out the Stereo matching based on stationary window, and utilize the disparity constraint based on color images to limit disparity search scope, obtain reference picture A ₁disparity map.As shown in Figure 5, Fig. 5 is a) the initial parallax figure adopting distance-like to estimate SAD acquisition, Fig. 5 b) be the initial parallax figure adopting associated class to estimate NCC acquisition.

Above-described embodiment is only be described the preferred embodiment of the present invention; not scope of the present invention is limited; under not departing from the present invention and designing the prerequisite of spirit; the various distortion that those of ordinary skill in the art make technical scheme of the present invention and improvement, all should fall in protection domain that claims of the present invention determines.

Claims (4)

1. fast substep a solid matching method, it is characterized in that, described matching process comprises the following steps:

(1) input the coloured image of the same scene that two width are taken under different angles, will wherein piece image as with reference to image A ₁, another piece image is as target image B ₁, suppose that two width images are corrected through polar curve all;

(2) respectively to reference picture A ₁with target image B ₁carry out the greyscale transformation based on local grain characteristic, obtain gray level image to A ₂and B ₂;

(3) utilize mean shift algorithm to reference picture A ₁carry out color images, adopt zero-mean normalized crosscorrelation metric function, using the cut zone of arbitrary size and shape as support window to gray level image to A ₂and B ₂carry out Stereo matching, the Matching power flow value in computed segmentation region, Matching power flow value according to cut zone is classified to each color segmentation region, when cut zone Matching power flow is more than or equal to 0.5, cut zone is categorized as reliable matching region, be labeled as 1, when cut zone Matching power flow is less than 0.5, cut zone is categorized as unreliable matching area, is labeled as 0, estimate the disparity range of pixel in regional according to classification results, form the disparity constraint based on color images;

(4) associated class is adopted to estimate normalized crosscorrelation metric function (Normalized cross-correlation respectively, NCC) and distance-like estimate the absolute value of gray scale difference and metric function (Sum of AbsoluteDifference, SAD) to gray level image to A ₂and B ₂carry out the Stereo matching based on stationary window, and utilize the disparity constraint based on color images to limit disparity search scope, obtain reference picture A ₁disparity map.

2. one according to claim 1 substep solid matching method fast, it is characterized in that, in step (2), described greyscale transformation, refer to and utilize the local grain contrast value of the texture features represented between pixel to carry out greyscale transformation to image, concrete computation process is as follows:

First, centered by object pixel p, get the mapping window W that a size is m × n, calculate the local grain contrast value Δ m of all pixels in this window, gray-scale value I (p) of center pixel is changed into I (p)+Δ m; Wherein, m is the length of mapping window, and n is the wide of mapping window;

Then, the size of the center pixel gray-scale value after comparing each neighborhood territory pixel and changing, sues for peace all pixel grey scales being greater than center pixel, by summed result assignment to center pixel.

3. one according to claim 1 substep solid matching method fast, it is characterized in that, the concrete computation process of described step (3) is as follows:

Define any pair matching area (s _p, s _q) when suppose parallax be d cut zone Matching power flow be E (s _p, s _q, d):

E(s _p,s _q,d)＝C _zncc(p,q,d)

$C_{\mathrm{zncc}}(p,q,d)=\frac{\underset{p\&Element;s_q,q\&Element;s_q}{\text{\&Sigma;}}(p-\stackrel{\&OverBar;}{p})\&CenterDot;(q-\stackrel{\&OverBar;}{q})}{\sqrt{\underset{p\&Element;s_p}{\mathrm{\&Sigma;}}{(p-\stackrel{\&OverBar;}{p})}^2\underset{q\&Element;s_q}{\mathrm{\&Sigma;}}{(q-\stackrel{\&OverBar;}{q})}^2}}$

Wherein, s _pit is reference picture any one cut zone after color segmentation; s _qregion corresponding with it in target image, i.e. s _p=s _q-d; D is the parallax value of hypothesis; P is cut zone s _pin any one pixel; Q is region s _qin a corresponding with it pixel, i.e. p ∈ s _p, q ∈ s _qand p=q-d; C _zncc(p, q, d) is cut zone s _pand corresponding region s _qsimilarity, adopt zero-mean normalized crosscorrelation metric function (Zero mean normalized cross-correlation, ZNCC) to estimate this similarity, and it can be used as the Matching power flow E (s of cut zone _p, s _q, d); cut zone s _pin the average of all grey scale pixel values; cut zone s _qin the average of all grey scale pixel values;

Utilize local optimization methods to maximize cut zone Matching power flow and obtain parallax value d _s, as the estimating disparity of pixels all in cut zone, that is:

d _s＝maxE(s _p,s _q,d)；d∈[d _min,d _max]

In formula, d _minminimum parallax value, d _maxit is maximum disparity value;

Classify to the result of cut zone Matching power flow, principle of classification is as cut zone Matching power flow E (s _p, s _q, when d)>=0.5, cut zone s _pbe classified as reliable matching region, be designated as 1; As cut zone Matching power flow E (s _p, s _q, d) during < 0.5, cut zone s _pbe classified as unreliable matching area, be designated as 0;

$s_p=\left\{\begin{array}{c}1,E(s_p,s_q,d)\&GreaterEqual;0.5\\ 0,E(s_p,s_q,d)<0.5\end{array}\right.$

Estimate that the disparity range of pixel in regional forms the disparity constraint of substep coupling according to classification results, constraint criterion is defined as:

d(s _p)∈[down,up]

In formula, down is the minimum value of carrying out disparity range after disparity constraint, and up is the maximal value of carrying out disparity range after disparity constraint;

Work as s _p=1time,

$\left\{\begin{array}{c}\mathrm{down}\&Element;\max (0,d_s-2)\\ \mathrm{up}\&Element;\min (d_{\max },d_s+2)\end{array}\right.$

Work as s _p=0time,

$\left\{\begin{array}{c}\mathrm{down}\&Element;\max (0,d_s-0.5\&times;d_{\max })\\ \mathrm{up}\&Element;\min (d_{\max },d_s+0.5\&times;d_{\max })\end{array}\right.$

Wherein d (s _p) be cut zone s _pin all pixel parallax spans.

4. one according to claim 2 substep solid matching method fast, it is characterized in that, local grain contrast value Δ m represents the texture features between pixel, and its computing method are:

Add up be designated as sum being greater than I (the p)+gray-scale value of Δ m in window W ₁, number is n ₁; Be less than I (the p)+gray-scale value of Δ m to add up and be designated as sum ₂, number is n ₂, then have:

other.