CN102136136A - Luminosity insensitivity stereo matching method based on self-adapting Census conversion - Google Patents

Abstract

The invention discloses a photometric insensitive stereo matching method based on adaptive Census transformation. First, an adaptive area based on a cross skeleton is determined according to the structure and color information of the image, so as to obtain a Census transformation window of any shape and size ;Secondly, using the Hamming distance after Census transformation as the matching cost, the local optimization method is used to calculate the initial disparity; finally, a two-step refinement method based on disparity statistical histogram and left-right consistency check is proposed, which will be based on the cross skeleton The self-adaptive area is organically integrated into the refinement process to obtain a high-precision disparity map. The present invention can obtain high-precision disparity maps for stereo image pairs with differences in illumination intensity and exposure time, taking into account matching accuracy and robustness to amplitude distortion, and can better adapt to the application scene of unmanned aerial vehicle visual navigation.

Description

The insensitive solid matching method of luminosity based on self-adaptation Census conversion

Technical field

The present invention relates to the solid matching method in the stereo visual system, belong to computer vision field, be used for existing under the condition of illumination and difference in exposure, obtain high-precision dense parallax information at left and right sides view, thereby for recovering to provide reliable assurance based on the three-dimensional depth information of stereoscopic vision.

Background technology

Three-dimensional coupling is a vital task in the computer vision field, and it obtains dense disparity map by binocular or many orders images match, thus the three-dimensional depth information in the perception scene.Domestic and international many scholars further investigate this field.Current dense Stereo Matching Algorithm can be divided into the coupling cost calculating, the coupling cost accumulation, calculating/optimizations of parallax, parallax carry the essence four steps analyze and research.In these four steps, the calculating of coupling cost is as the basis of solid coupling, and its significance level is self-evident, has only the suitable coupling cost of selection could obtain high-precision disparity map, and common coupling cost can be divided into two classes:

First kind coupling cost is based on the hypothesis of brightness/color consistency, be that feature in the scene has identical brightness/color information in different images, as the absolute value of gray scale difference, gray scale difference square, the gray scale difference absolute value that blocks etc. all is based on this hypothesis.

Yet, owing to there is the influence of factors such as global brightness variation, local brightness variation and noise between image, cause the brightness/color information of character pair inequality, this phenomenon is referred to as amplitude distortion.Although it is the matching algorithm based on brightness/color consistency can obtain high-precision disparity map for the image that satisfies this hypothesis, quite responsive for amplitude distortion.

Another kind of coupling cost reaches the insensitive purpose of amplitude distortion by hypothesis lax or that abandon brightness/color consistency, as normalized crosscorrelation, Rank and the variation of Census nonparametric, mutual information and Laplce's gaussian sum medium filtering etc.Studies show that the Census non-parametric transformations all has good robustness under the condition of various amplitude distortions.But there is the selection problem of window size in tradition based on the Census conversion of stationary window.If mapping window is too little, then signal to noise ratio (S/N ratio) is low excessively, and it is low to cause mating the cost discrimination, easily low texture region is caused the mistake coupling; If mapping window is excessive, then can introduce too much outlier (Outlier), influence matching precision.

In addition, research also shows by the coupling cost of global approach optimization based on the Census conversion, will cause high calculation cost; Utilization will be difficult to obtain high-precision anaglyph based on the partial approach optimization of the fixing support window coupling cost based on the Census conversion, make the parallax discontinuity zone exist tangible prospect to amplify phenomenon.

Summary of the invention

Technical matters to be solved by this invention provides a kind of solid matching method, exists at left and right sides view under the condition of illumination and difference in exposure to obtain high-precision anaglyph.

For solving the problems of the technologies described above, the present invention takes following technical scheme to realize:

A kind of insensitive solid matching method of luminosity based on self-adaptation Census conversion is characterized in that may further comprise the steps:

(1) definite adaptive region based on crossing skeleton obtains the Census mapping window of arbitrary shape and size with this;

(2) utilize Hamming distance after the Census conversion as the coupling cost, adopt local optimization methods (Winner-Take-all) to calculate initial parallax;

(3) behind the acquisition initial parallax, utilize two steps of parallax to put forward smart method and further improve the parallax precision.

The aforementioned insensitive solid matching method of luminosity based on self-adaptation Census conversion is characterized in that the coupling cost in the three-dimensional matching process is calculated and the polymerization of coupling cost is organically merged.

The aforementioned insensitive solid matching method of luminosity based on self-adaptation Census conversion is characterized in that the preparation method of the Census mapping window of described arbitrary shape and size comprises the steps:

(1) makes up in reference picture and the target image each pixel respectively based on the adaptive region of crossing skeleton according to the structure of image and color information;

(2) adaptive region, the adaptive region of corresponding point in target image in reference picture to be matched obtained the Census mapping window by logic and operation.

The aforementioned insensitive solid matching method of luminosity based on self-adaptation Census conversion is characterized in that the specific algorithm of described adaptive region based on crossing skeleton is:

(1) any one pixel p in the image is determined the skeleton of a cross shape, this skeleton has comprised level and vertical both direction, uses H (p) and V (p) expression respectively, and the length of the four direction of skeleton can be expressed as (h _p ^-, h _p ⁺, v _p ^-, v _p ⁺), pixel p can be expressed as based on the adaptive region of crossing skeleton:

$U\left(p\right)=\underset{q\∈V\left(p\right)}{\∪}H\left(q\right)---\left(1\right)$

(2) according to the hypothesis of the corresponding same structure of similar color in the image, adopt following formula:

$r^{*}=\underset{r\∈[1,L]}{\max }\left(r\underset{i\∈[1,r]}{\mathrm{\Π}}\mathrm{\δ}(p,p_i)\right)---\left(2\right)$

Determine the length (h of center pixel p crossing skeleton four direction respectively _p ^-, h _p ⁺, v _p ^-, v _p ⁺), in the formula (2), δ is an indicator function, is used for weighing the heterochromia degree between different pixels, p _iBe a pixel on the p cross direction, the coordinate representation of p in image is (x _p, y _p), r ^*Be the brachium of certain cross direction, L is along the hunting zone of p cross direction, works as p _iAt the level left side of p, then p _iCoordinate in image can be expressed as (x _p-i, y _p), L is the hunting zone along the horizontal left direction of pixel p, then r ^*Result of calculation be h _p ^-, in like manner determine the length h of other three directions _p ⁺, v _p ^-, v _p ⁺

The aforementioned insensitive solid matching method of luminosity based on self-adaptation Census conversion is characterized in that described two steps based on parallax statistic histogram and left and right sides consistency desired result carry precision method and comprise the steps:

(1) according to the hypothesis of parallax smooth change in similar color area, in the adaptive region that can reflect picture structure and color information, initial parallax is carried out putting forward essence based on the first step of statistic histogram;

(2) in order to get rid of insincere parallax, the horizontal parallax figure that the first step is carried after the essence carries out left and right sides consistency desired result, set up parallax and put the letter matrix, in adaptive region, credible parallax is adopted optimization based on the parallax statistic histogram subsequently, get rid of the influence of occlusion area and insincere parallax statistics.

The aforementioned insensitive solid matching method of luminosity based on self-adaptation Census conversion, it is characterized in that described parallax statistic histogram optimization specific implementation method is: for any one pixel to be matched, statistics its based on the adaptive region of crossing skeleton in the frequency of occurrence of different parallaxes, select to have the parallax value of maximum generate probability as optimizing the result.

So far, the insensitive solid matching method of luminosity based on self-adaptation Census conversion is finished.

The beneficial effect of patent of the present invention is: the present invention has not only kept the robustness of Census conversion for amplitude distortion, and do not adopting under the prerequisite of global optimization, solved existing solid matching method is difficult to obtain high-precision disparity map under the prerequisite of left and right sides view existence exposure and illumination difference problem, can better adapt to true navigation scenarios based on stereoscopic vision.

Description of drawings

Fig. 1 is an algorithm flow chart of the present invention;

Fig. 2 is the schematic diagram based on the adaptive region of crossing skeleton;

Fig. 3 carries smart process flow diagram for second step of parallax.

Embodiment

Below in conjunction with accompanying drawing and example patent of the present invention is further specified.

As shown in Figure 1, a kind of quick stereo matching process of cutting apart with self-adapting window based on color may further comprise the steps:

The first step: determine a kind of adaptive region, obtain the Census mapping window of arbitrary shape and size with this based on crossing skeleton.

1. as shown in Figure 2, determine that at first each pixel is based on the adaptive region of crossing skeleton in reference picture and the target image.Any one pixel p in the image is determined the skeleton of a cross shape, and this skeleton has comprised level and vertical both direction, uses H (p) and V (p) expression respectively, and pixel p can be expressed as based on the adaptive region of crossing skeleton:

$U\left(p\right)=\underset{q\∈V\left(p\right)}{\∪}H\left(q\right)---\left(1\right)$

The length of the four direction of skeleton can be expressed as (h _p ^-, h _p ⁺, v _p ^-, v _p ⁺); Hypothesis according to the corresponding same structure of similar color in the image, adopt following formula:

$r^{*}=\underset{r\∈[1,L]}{\max }\left(r\underset{i\∈[1,r]}{\mathrm{\Π}}\mathrm{\δ}(p,p_i)\right)---\left(2\right)$

Determine the length (h of center pixel p crossing skeleton four direction respectively _p ^-, h _p ⁺, v _p ^-, v _p ⁺), in the formula (2), δ is an indicator function, is used for weighing the heterochromia degree between different pixels, p _iBe a pixel on the p cross direction, the coordinate representation of p in image is (x _p, y _p), r ^*Be the brachium of certain cross direction, L is (L=17 in the experiment) along the hunting zone of p cross direction, works as p _iAt the level left side of p, then p _iCoordinate in image can be expressed as (x _p-i, y _p), L is the hunting zone along the horizontal left direction of pixel p, then r ^*Result of calculation be h _p ^-, in like manner determine the length h of other three directions _p ⁺, v _p ^-, v _p ⁺

2. at last, determine the Census mapping window of any pixel.If U _Ref(m) and U _Tar(n) be illustrated respectively in corresponding point m under the parallax hypothesis d, n is based on the adaptive region of crossing skeleton, m then, and the Census mapping window of the arbitrary shape of n in correspondence image and size can be expressed as U _d(m) and U _d(n):

U _d(m)＝{(x，y)|(x，y)∈U _ref(m)，(x-d，y)∈U _tar(n)}(2)

U _d(n)＝{(x，y)|(x，y)∈U _tar(n)，(x+d，y)∈U _ref(m)}(3)

Because m=(x, y), n=(x-d, y), so U _d(m) and U _d(n) be of similar shape and size, and make N that (m n) represents its size.

Second step: utilize Hamming distance after the Census conversion as the coupling cost, adopt local optimization methods to calculate initial parallax.This process will mate that cost is calculated and the coupling cost is accumulated two steps and organically blended, be because previous step has been selected suitable Census mapping window according to the structure and the color information of image, this window has good adaptive to low texture region, parallax discontinuity zone etc. in the image; If further adopt stationary window that the coupling cost of Census conversion is carried out polymerization, not only can cause facing a difficult choice of low texture region and parallax discontinuity zone coupling again, also can increase suitable calculated amount.

The 3rd step: after obtaining initial parallax, propose a kind of two steps of parallax to put forward smart method and further improve the parallax precision.

1. in the local optimum process, owing to will mate the calculating of cost and the accumulation of coupling cost organically blends, thus can there be certain noise in the initial parallax, and second step was carried left and right sides consistency desired result in the smart process to the suitable sensitivity of noise.For this reason, according to the hypothesis of parallax smooth change in similar color area, employing can reflect the U of picture structure and color information _RefInitial parallax is carried out the first step put forward essence.As shown in Figure 1, to any pixel m in the reference picture, set up the statistic histogram of an initial parallax

Statistics U _Ref(m) frequency that all different initial parallaxes occur in, and select

Peak value carry parallax d as a result after the essence as the first step ₁, can be expressed as:

d∈[d _min，d _max] (4)

2. in order further to detect the occlusion area in the anaglyph, get rid of insincere parallax, carried out for second step by Fig. 3 flow process and carry essence, wherein d _1LAnd d _1RRepresent that respectively with left figure and right figure be parallax after the first step that reference picture obtains is put forward essence, U _LeftAnd U _RightRepresent that respectively each pixel is based on the adaptive region of crossing skeleton among left figure and the right figure.

At first, adopt left and right sides consistency desired result to detect d _1LAnd d _1RThe degree of confidence of middle parallax is set up parallax and is put the letter matrix, if | d _1L(x, y)-d _1R(x+d _1L(x, y), y) |＜T (in the experiment, T is 1 pixel), think that then parallax is credible, on the contrary then insincere; Secondly, select d _1LAnd d _1RIn any one further handle and (in the experiment, selected d _1L), processing mode still is based on the parallax statistic histogram, but only adds up U this moment _LeftIn credible parallax, thereby can get rid of occlusion area and insincere parallax for the influence of statistics, further improve the parallax precision.

In sum, be difficult under the true vision guided navigation scene of amplitude distortion to obtain the high precision parallax at existing Stereo Matching Algorithm, the present invention proposes and a kind ofly will mate that cost is calculated and the coupling cost is accumulated the local matching process based on self-adaptation Census conversion that organically blends.Experimental result shows that this algorithm can obtain more high-precision disparity map to the left and right sides view that has intensity of illumination and time shutter difference, has taken into account matching precision and to the robustness of amplitude distortion, can better adapt to the application scenarios of unmanned plane vision navigation.

Above-mentioned embodiment does not limit technical scheme of the present invention in any form, and the technical scheme that mode obtained that every employing is equal to replacement or equivalent transformation all drops on protection scope of the present invention.

Claims (5)

1. a photometric insensitive stereo matching method based on adaptive Census transform, is characterized in that comprising the following steps: (1) Determine a kind of adaptive region based on the cross skeleton, so as to obtain a Census transformation window of any shape and size; (2) Use the Hamming distance after Census transformation as the matching cost, and use the local optimization method to calculate the initial disparity; (3) After obtaining the initial disparity, use the disparity two-step refinement method to further improve the disparity accuracy. 2. the photometric insensitive stereo matching method based on adaptive Census transformation according to claim 1, is characterized in that: the obtaining method of the Census transformation window of described arbitrary shape and size comprises the steps: (1) According to the structure and color information of the image, respectively construct the self-adaptive area based on the cross skeleton for each pixel in the reference image and the target image; (2) Obtain the Census transform window by logical AND operation of the adaptive area of the point to be matched in the reference image and the adaptive area of the corresponding point in the target image. 3. the photometric insensitive stereo matching method based on adaptive Census transformation according to claim 2, is characterized in that: the specific algorithm of the adaptive region based on the described cross skeleton is: (1) Determine a cross-shaped skeleton for any pixel p in the image. The skeleton includes two directions, horizontal and vertical, denoted by H(p) and V(p) respectively. The lengths of the four directions of the skeleton can be expressed As (h _p ^- , hp ₊ ^, v _p ^- , v _p ⁺ ), the adaptive area of pixel p based on the cross skeleton can be expressed as: $\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; V</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span>}{<span\; class="notranslate">\; \&cup;</span>}\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$ (2) According to the assumption that similar colors in the image correspond to the same structure, the following formula is adopted: ${\mathrm{<span\; class="notranslate">\; r</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; \&Element;</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; ]</span>}{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; r</span>}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&Element;</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ]</span>}{\mathrm{<span\; class="notranslate">\; \&Pi;</span>}}\mathrm{<span\; class="notranslate">\; \&delta;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$ Determine the lengths (h _p ^- , h _p ⁺ , v _p ^- , v _p ⁺ ) of the four directions of the cross skeleton of the central pixel p respectively. In formula (2), δ is an indicator function used to measure the color between different pixels The degree of difference, p _i is a pixel in the cross direction of p, the coordinates of p in the image are expressed as (x _p , y _p ), r ^* is the arm length in a certain cross direction, L is the arm length along a certain cross direction of p Search range, when p _i is on the horizontal left side of p, then the coordinates of p _i in the image can be expressed as (x _p -i, y _p ), L is the search range along the horizontal left side of pixel p, then r The calculation result of ^* is h _p ^- , and the lengths h _p ⁺ , v _p ^- , and v _p ⁺ of the other three directions are determined similarly. 4. the photometric insensitive stereo matching method based on adaptive Census transformation according to claim 1, is characterized in that: the described two-step refinement method based on parallax statistics histogram and left and right consistency checking comprises the steps: (1) According to the assumption that the disparity changes smoothly in similar color regions, the initial disparity is refined in the first step based on the statistical histogram in the adaptive region that can reflect the image structure and color information; (2) In order to eliminate unreliable disparity, the left and right disparity images refined in the first step are checked for left and right consistency, and the disparity confidence matrix is established, and then the credible disparity is optimized based on the disparity statistical histogram in the adaptive area , to exclude the influence of occluded regions and unreliable parallax on the statistical results. 5. the photometric insensitive stereo matching method based on adaptive Census transform according to claim 4, it is characterized in that described disparity statistical histogram optimization specific implementation method is: for any pixel to be matched, statistics is based on cross The occurrence frequency of different disparities in the adaptive area of the skeleton, and the disparity value with the largest occurrence probability is selected as the optimization result.

Images