CN105141937A - BEMD (Bidirectional Empirical Mode Decomposition) based self-adaptive stereo disparity estimation method - Google Patents

Abstract

The invention discloses a BEMD-based self-adaptive stereo disparity estimation method. The disparity is estimated according to phase information of an image, the image structure tends to be stable with change of external illumination and the like, and the matching accuracy and robustness are ensured. The image is deposited into single component information of different frequency bands adaptively on the basis of EMD, effective phase information is extracted, and the selection problem of extracting phase medium and low wavelet base on the basis of wavelet transformation is avoided. The result accuracy and robustness obtained on the basis of EMD information matching can be seen from a matching result graph based on image gray scale information and EMD phase information.

Description

A kind of adaptive three-dimensional parallax estimation method based on Bidimensional Empirical Mode Decomposition

[technical field]

The invention belongs to technical field of visual measurement, relate to a kind of based on Bidimensional Empirical Mode Decomposition (BEMD) algorithm for estimating, specifically a kind of adaptive three-dimensional parallax estimation method based on Bidimensional Empirical Mode Decomposition.

[background technology]

Disparity estimation is the Focal point and difficult point that stereo vision three-dimensional depth information recovers.Current Disparity estimation is mainly divided into two large class, the disparity estimation based on gradation of image information and the disparity estimation based on image structure information.

Disparity estimation based on gradation of image information only relies on image luminance information, affects larger by illumination variation, block etc.; Picture structure mainly refers to the information such as local phase, partial amplitudes, and because phase place change is the quantifiable continuous variable independent of image space, the parallax estimation method based on phase place can be easy to reach subpixel accuracy.Picture structure, along with ambient light is according to waiting change generally more stable, means that local phase is substantially constant, ensure that the robustness of disparity estimation.

Disparity estimation based on phase place needs to adopt suitable wavelet basis extraction image local phase place and amplitude information as disparity estimation primitive.As the disparity estimation based on Gabor filtering (StereodisparitycomputationusingGaborfilters) that TD.Sanger proposes, XHuang etc. propose based on continuous wavelet transform (Continuouswavelettransformation, CWT) disparity estimation (Densedisparityestimationbasedonthecontinuouswavelettrans form [stereoimageanalysis]), SKumar etc. are based on two tree quadtrees small echo (Dualtreequaternionwavelettransform, DTQWT) disparity estimation (Dualtreefractionalquaternionwavelettransformfordisparity estimation) and based on dual-tree complex wavelet (Dualtreecomplexwavelettransform, DTCWT) etc.

The Major Difficulties faced at present based on the disparity estimation of phase place how to select suitable wavelet basis to carry out local phase extraction and analysis according to image spatial feature, and different small echos is selected larger to Influence on test result.Bidimensional Empirical Mode Decomposition (BEMD) is a kind of signal processing method of full-automatic data-driven, there is local auto-adaptive, do not need predefined filter or wavelet function, in process non-stationary signal and nonlinear properties, there is obvious advantage.

[summary of the invention]

The object of the invention is to solve the problem, a kind of adaptive three-dimensional parallax estimation method based on Bidimensional Empirical Mode Decomposition is provided, the method both can play high accuracy and the robustness of phase place disparity estimation, also can evade a Selection of Wavelet Basis difficult problem, BEMD is applied to disparity estimation field by the present invention first simultaneously.

For achieving the above object, the technical solution adopted in the present invention comprises the following steps:

Step 1: left image L (x, y) and right image R (x, y) are sieved into a series of Intrinsic mode functions and residual components based on BEMD self adaptation respectively;

Step 2: utilize the difference of Poisson kernel core to carry out bandpass filtering to the IMF of left image and right image and residual components, and obtain corresponding two-dimensional analysis signal by plural Riesz conversion;

Step 3: local phase Φ, partial amplitudes Α and the instantaneous frequency of extracting two-dimensional analysis signal

Step 4: carry out initial parallax estimation according to the local phase extracted and instantaneous frequency information at each IMF yardstick:

Step 5: final parallax gets the weighted average of the parallax that each yardstick obtains; Weight coefficient covers amplitude coupling index, different scale estimating disparity goodness of fit index and slickness hypothesis index.

The present invention further improves and is:

In step 1, Intrinsic mode functions is:

$I(x,y)={\mathrm{\Σ}}_{j=1}^{j=\mathrm{\ζ}}{\mathrm{IMF}}_j^I(x,y)+{\mathrm{Residue}}_j^I(x,y)(I=L,R)---\left(1\right)$

In step 2, utilize the difference of Poisson kernel core to carry out bandpass filtering to the IMF of left images, and obtain corresponding two-dimensional analysis signal by plural Riesz conversion; Yardstick is the expression formula of Poisson kernel core in spatial domain of s

$\mathrm{\κ}(s,x,y)=\frac{s}{2\mathrm{\π}{(s^2+x^2+y^2)}^{3/2}}---\left(2\right)$

The plural Riesz conversion of its correspondence is as follows in the expression formula in spatial domain, and formula (3) and formula (4) represent its real part and imaginary part respectively:

$real\[R_{\mathrm{\κ}}(s_1,s_2,x,y)\]=\frac{\mathrm{\κ}(s_1,x,y)\·x}{s_1\mathrm{\Σ}\mathrm{\κ}(s_1,x,y)}-\frac{\mathrm{\κ}(s_2,x,y)\·x}{s_2\mathrm{\Σ}\mathrm{\κ}(s_2,x,y)}---\left(3\right)$

$imag\[R_{\mathrm{\κ}}(s_1,s_2,x,y)\]=\frac{\mathrm{\κ}(s_1,x,y)\·y}{s_1\mathrm{\Σ}\mathrm{\κ}(s_1,x,y)}-\frac{\mathrm{\κ}(s_2,x,y)\·y}{s_2\mathrm{\Σ}\mathrm{\κ}(s_2,x,y)}---\left(4\right)$

The IMF of left images and residual components carry out plural Riesz conversion:

f _R1＝f*real(R _κ)(5)

f _R2＝f*imag(R _κ)(6)

Wherein * represents convolution algorithm, and f represents primary signal, comprises IMF component and the screening residual components of left images here; { f, f _r1, f _r1form two dimension singly drill signal space.

In step 3 and 4, extract local phase Φ, partial amplitudes Α and the instantaneous frequency of two-dimensional analysis signal concrete grammar as follows:

$\mathrm{\Φ}(x,y)=arctan\frac{\sqrt{f_{R1}^2+f_{R2}^2}}{f^2}---\left(7\right)$

$A(x,y)=\sqrt{f_{R1}^2+f_{R2}^2+f^2}---\left(8\right)$

At each IMF yardstick and residual components according to local phase and instantaneous Frequency Estimation initial parallax figure:

In described step 5, final parallax gets the weighted average of the parallax that each yardstick obtains, and weight coefficient covers amplitude coupling index, different scale estimating disparity goodness of fit index and slickness hypothesis index, and the acquisition process of final parallax is as follows:

A () often puts the amplitude coupling weight coefficient γ of parallax at the first dimension calculation ₁, γ ₁∈ (0,1), γ ₁larger, illustrate that amplitude information mates better, the parallax of estimation is more reliable:

${\mathrm{\γ}}_1(x,y)=min\[\frac{\left|A_R(x,y)\right|}{\left|A_L(x+d,y)\right|},\frac{\left|A_L(x+d,y)\right|}{\left|A_R(x,y)\right|}\]---\left(11\right)$

B at different scale, () estimates that the parallax of a certain picture position should be identical, the goodness of fit is higher, and the more accurate of disparity estimation is described; The weighting parallax obtained at the first dimension calculation and next yardstick are compared, obtains goodness of fit coefficient gamma ₂:

${\mathrm{\γ}}_2(x,y)=\frac{{\mathrm{\Σ\γ}}_1(x,y)}{{\mathrm{\Σ\γ}}_1(x,y)+\mathrm{\Σ}\[{\mathrm{\γ}}_1(x,y)\·|d_2(x,y)-d_1(x,y)|\]}---\left(12\right)$

γ ₂∈ (0,1), γ ₂the parallax goodness of fit that larger explanation different scale is estimated is better, and parallax precision is higher;

C () continues to compare the parallax goodness of fit between the first yardstick and follow-up 3rd yardstick, the 4th yardstick etc., obtain the coefficient of reliability that different scale is often put;

D () obtains reliable disparity map by the parallax value weighted average of different scale:

$d(x,y)=\frac{{\mathrm{\Σ}}_{scale}\[d_{scale}(x,y)\·{\mathrm{\γ}}_{2,scale}(x,y)\]}{{\mathrm{\Σ}}_{scale}{\mathrm{\γ}}_{2,scale}(x,y)}---\left(13\right).$

In described step 5, the acquisition process of final parallax is further comprising the steps of:

The reprocessing of (e) disparity map:

Suppose that neighbor point should have close parallax, the final parallax of each point is the weighted average of itself and neighborhood parallax value, and weight coefficient is determined by the coefficient of reliability of each point.

Compared with prior art, the present invention has following beneficial effect:

The present invention adopts the phase information of image to carry out disparity estimation, and picture structure, along with ambient light is according to waiting change generally more stable, ensure that accuracy and the robustness of coupling.The present invention is based on the simple component information that picture breakdown is different frequency bands by EMD adaptively, extract effective phase information, evade the selection difficult problem extracting wavelet basis in phase place at present based on wavelet transformation.The present invention both can play high accuracy and the robustness of phase place disparity estimation, also can evade a Selection of Wavelet Basis difficult problem, and BEMD is applied to disparity estimation field by the present invention first simultaneously.

[accompanying drawing explanation]

Fig. 1 is Implementation Roadmap of the present invention;

Fig. 2 is the image that the embodiment of the present invention uses; Wherein, a is left image, and b is right image;

Fig. 3 is the EMD discomposing effect figure in the embodiment of the present invention; Wherein, a-d is respectively first three IMF component of left image EMD decomposition and decomposes residual volume, and e-h is respectively first three IMF component and decomposition residual volume that right image is EMD decomposition;

Fig. 4 is result and the Riesz transformation results figure of first IMF bandpass filtering; Wherein, a-c represents real part and the imaginary of the bandpass filtering result of left image first IMF and Riesz conversion respectively, and d-f represents the bandpass filtering result of right image first IMF and the real part of Riesz conversion and imaginary respectively;

Fig. 5 is based on half-tone information and the disparity estimation result figure based on EMD phase information.

[embodiment]

Below in conjunction with accompanying drawing, the present invention is described in further detail:

See Fig. 1, the present invention includes following steps:

Step 1: left image L (x, y) and right image R (x, y) are sieved into a series of Intrinsic mode functions (Intrinsicmodefunction, IMF) and residual components based on BEMD self adaptation respectively.Screening process is actually process multicomponent data processing being decomposed into simple component signal.The IMF first sieving out has the meticulousst dimensional information, then the like.In order to illustrate, be decomposed into 3 IMF to left images as an example, decomposition result is shown in Fig. 3, and the composition of visible signal different frequency is broken down in different IMF components, is convenient to the extraction of instantaneous frequency and phase place.

$I(x,y)={\mathrm{\Σ}}_{j=1}^{j=\mathrm{\ζ}}{\mathrm{IMF}}_j^I(x,y)+{\mathrm{Residue}}_j^I(x,y)(I=L,R)---\left(1\right)$

Step 2: utilize the difference of Poisson kernel core to carry out bandpass filtering to the IMF of left images, is shown in Fig. 4 (a) & (d) and obtains corresponding two-dimensional analysis signal by plural Riesz conversion.2D signal and Riesz conversion thereof are formed singly drills signal, has rotational invariance.Yardstick is the expression formula of Poisson kernel core in spatial domain of s

$\mathrm{\κ}(s,x,y)=\frac{s}{2\mathrm{\π}{(s^2+x^2+y^2)}^{3/2}}---\left(2\right)$

The plural Riesz conversion of its correspondence is as follows in the expression formula in spatial domain, and (3), (4) represent its real part and imaginary part respectively:

$real\[R_{\mathrm{\κ}}(s_1,s_2,x,y)\]=\frac{\mathrm{\κ}(s_1,x,y)\·x}{s_1\mathrm{\Σ}\mathrm{\κ}(s_1,x,y)}-\frac{\mathrm{\κ}(s_2,x,y)\·x}{s_2\mathrm{\Σ}\mathrm{\κ}(s_2,x,y)}---\left(3\right)$

$imag\[R_{\mathrm{\κ}}(s_1,s_2,x,y)\]=\frac{\mathrm{\κ}(s_1,x,y)\·y}{s_1\mathrm{\Σ}\mathrm{\κ}(s_1,x,y)}-\frac{\mathrm{\κ}(s_2,x,y)\·y}{s_2\mathrm{\Σ}\mathrm{\κ}(s_2,x,y)}---\left(4\right)$

The IMF of left images and residual components carry out plural Riesz conversion, the results are shown in Figure 4 (b)-(c), (e)-(f), and the result of visible Riesz conversion can the partial structurtes information of reflection image of robust:

f _R1＝f*real(R _κ)(5)

f _R2＝f*imag(R _κ)(6)

Wherein * represents convolution algorithm, and f represents primary signal, comprises IMF component and the screening residual components of left images here.{ f, f _r1, f _r1form two dimension singly drill signal space.

Step 3: local phase Φ, partial amplitudes Α, the instantaneous frequency of extracting two-dimensional analysis signal

$\mathrm{\Φ}(x,y)=arctan\frac{\sqrt{f_{R1}^2+f_{R2}^2}}{f^2}---\left(7\right)$

$A(x,y)=\sqrt{f_{R1}^2+f_{R2}^2+f^2}---\left(8\right)$

Wherein Δ represents the real part R of the rear analytic signal of Riesz conversion ₁with imaginary part R ₂, ▽ represents local derviation.The solution utilizing formula (9) to ask instantaneous frequency to avoid when directly calculating phase place wraps up problem.

Step 4: at each IMF yardstick and residual components according to local phase and instantaneous Frequency Estimation initial parallax figure:

Step 5: final parallax gets the weighted average of the parallax that each yardstick obtains.Weight coefficient covers amplitude coupling index, different scale estimating disparity goodness of fit index and slickness hypothesis index.The acquisition process of final parallax is as follows:

(a) due to amplitude comparatively large with condition variable effects such as ambient light photographs, do not participate in disparity estimation, but can as the index weighing disparity estimation precision.The amplitude coupling weight coefficient often putting parallax at the first dimension calculation is as follows, γ ₁∈ (0,1), γ ₁larger, illustrate that amplitude information mates better, the parallax of estimation is more reliable.

${\mathrm{\γ}}_1(x,y)=min\[\frac{\left|A_R(x,y)\right|}{\left|A_L(x+d,y)\right|},\frac{\left|A_L(x+d,y)\right|}{\left|A_R(x,y)\right|}\]---\left(11\right)$

B at different scale, () estimates that the parallax of a certain picture position should be identical in theory, the goodness of fit is higher, and the more accurate of disparity estimation is described.The weighting parallax obtained at the first dimension calculation and next yardstick are compared, obtain goodness of fit coefficient, adopt following form:

${\mathrm{\γ}}_2(x,y)=\frac{{\mathrm{\Σ\γ}}_1(x,y)}{{\mathrm{\Σ\γ}}_1(x,y)+\mathrm{\Σ}\[{\mathrm{\γ}}_1(x,y)\·|d_2(x,y)-d_1(x,y)|\]}---\left(12\right)$

γ ₂∈ (0,1), γ ₂the parallax goodness of fit that larger explanation different scale is estimated is better, and parallax precision is higher.D ₁(x, y), d ₂(x, y) represents the disparity estimation value of the first yardstick and the second yardstick respectively.

C () utilizes formula (12) to continue to compare the parallax goodness of fit between the first yardstick and follow-up 3rd yardstick, the 4th yardstick etc., obtain the coefficient of reliability that different scale is often put.

D () obtains reliable disparity map by the parallax value weighted average of different scale,

$d(x,y)=\frac{{\mathrm{\Σ}}_{scale}\[d_{scale}(x,y)\·{\mathrm{\γ}}_{2,scale}(x,y)\]}{{\mathrm{\Σ}}_{scale}{\mathrm{\γ}}_{2,scale}(x,y)}---\left(13\right)$

The reprocessing of (e) disparity map.Suppose that neighbor point should have close parallax, the final parallax of each point is the weighted average of itself and neighborhood parallax value.Weight coefficient is determined by the coefficient of reliability of each point.

Fig. 5 is based on gradation of image information with based on EMD phase information matching result figure, obtains accuracy and the robustness of result as seen based on EMD phase information coupling.

Above content is only and technological thought of the present invention is described; protection scope of the present invention can not be limited with this; every technological thought proposed according to the present invention, any change that technical scheme basis is done, within the protection range all falling into claims of the present invention.

Claims (6)

1., based on an adaptive three-dimensional parallax estimation method for Bidimensional Empirical Mode Decomposition, it is characterized in that, comprise the following steps:

Step 1: left image L (x, y) and right image R (x, y) are sieved into a series of Intrinsic mode functions and residual components based on BEMD self adaptation respectively;

Step 2: utilize the difference of Poisson kernel core to carry out bandpass filtering to the IMF of left image and right image and residual components, and obtain corresponding two-dimensional analysis signal by plural Riesz conversion;

Step 3: local phase Φ, partial amplitudes Α and the instantaneous frequency of extracting two-dimensional analysis signal

Step 4: carry out initial parallax estimation according to the local phase extracted and instantaneous frequency information at each IMF yardstick:

Step 5: final parallax gets the weighted average of the parallax that each yardstick obtains; Weight coefficient covers amplitude coupling index, different scale estimating disparity goodness of fit index and slickness hypothesis index.

2., as claimed in claim 1 based on the adaptive three-dimensional parallax estimation method of Bidimensional Empirical Mode Decomposition, it is characterized in that, in described step 1, Intrinsic mode functions is:

$I(x,y)={\mathrm{\Σ}}_{j=1}^{j=\mathrm{\ζ}}{\mathrm{IMF}}_j^I(x,y)+\mathrm{Re}{\mathrm{sidue}}_j^I(x,y),(I=L,R)---\left(1\right).$

3. as claimed in claim 1 based on the adaptive three-dimensional parallax estimation method of Bidimensional Empirical Mode Decomposition, it is characterized in that, in described step 2, utilize the difference of Poisson kernel core to carry out bandpass filtering to the IMF of left images, and obtain corresponding two-dimensional analysis signal by plural Riesz conversion; Yardstick is the expression formula of Poisson kernel core in spatial domain of s

$\mathrm{\κ}(s,x,y)=\frac{s}{2\mathrm{\π}{(s^2+x^2+y^2)}^{3/2}}---\left(2\right)$

The plural Riesz conversion of its correspondence is as follows in the expression formula in spatial domain, and formula (3) and formula (4) represent its real part and imaginary part respectively:

$real\[R_{\mathrm{\κ}}(s_1,s_2,x,y)\]=\frac{\mathrm{\κ}(s_1,x,y)\·x}{s_1\mathrm{\Σ}\mathrm{\κ}(s_1,x,y)}-\frac{\mathrm{\κ}(s_2,x,y)\·x}{s_2\mathrm{\Σ}\mathrm{\κ}(s_2,x,y)}---\left(3\right)$

$imag\[R_{\mathrm{\κ}}(s_1,s_2,x,y)\]=\frac{\mathrm{\κ}(s_1,x,y)\·y}{s_1\mathrm{\Σ}\mathrm{\κ}(s_1,x,y)}-\frac{\mathrm{\κ}(s_2,x,y)\·y}{s_2\mathrm{\Σ}\mathrm{\κ}(s_2,x,y)}---\left(4\right)$

The IMF of left images and residual components carry out plural Riesz conversion:

f _R1＝f*real(R _κ)(5)

f _R2＝f*imag(R _κ)(6)

Wherein * represents convolution algorithm, and f represents primary signal, comprises IMF component and the screening residual components of left images here; { f, f _r1, f _r1form two dimension singly drill signal space.

4. as claimed in claim 1 based on the adaptive three-dimensional parallax estimation method of Bidimensional Empirical Mode Decomposition, it is characterized in that, in described step 3 and 4, extract local phase Φ, partial amplitudes Α and the instantaneous frequency of two-dimensional analysis signal concrete grammar as follows:

$\mathrm{\Φ}(x,y)=arctan\frac{\sqrt{f_{R1}^2+f_{R2}^2}}{f^2}---\left(7\right)$

$A(x,y)=\sqrt{f_{R1}^2+f_{R2}^2+f^2}---\left(8\right)$

At each IMF yardstick and residual components according to local phase and instantaneous Frequency Estimation initial parallax figure:

5. as claimed in claim 1 based on the adaptive three-dimensional parallax estimation method of Bidimensional Empirical Mode Decomposition, it is characterized in that, in described step 5, final parallax gets the weighted average of the parallax that each yardstick obtains, weight coefficient covers amplitude coupling index, different scale estimating disparity goodness of fit index and slickness hypothesis index, and the acquisition process of final parallax is as follows:

A () often puts the amplitude coupling weight coefficient γ of parallax at the first dimension calculation ₁, γ ₁∈ (0,1), γ ₁larger, illustrate that amplitude information mates better, the parallax of estimation is more reliable:

${\mathrm{\γ}}_1(x,y)=min\[\frac{\left|A_R(x,y)\right|}{\left|A_L(x+d,y)\right|},\frac{\left|A_L(x+d,y)\right|}{\left|A_R(x,y)\right|}\]---\left(11\right)$

B at different scale, () estimates that the parallax of a certain picture position should be identical, the goodness of fit is higher, and the more accurate of disparity estimation is described; The weighting parallax obtained at the first dimension calculation and next yardstick are compared, obtains goodness of fit coefficient gamma ₂:

${\mathrm{\γ}}_2(x,y)=\frac{{\mathrm{\Σ\γ}}_1(x,y)}{{\mathrm{\Σ\γ}}_1(x,y)+\mathrm{\Σ}\[{\mathrm{\γ}}_1(x,y)\·|d_2(x,y)-d_1(x,y)|\]}---\left(12\right)$

γ ₂∈ (0,1), γ ₂the parallax goodness of fit that larger explanation different scale is estimated is better, and parallax precision is higher;

C () continues to compare the parallax goodness of fit between the first yardstick and follow-up 3rd yardstick, the 4th yardstick etc., obtain the coefficient of reliability that different scale is often put;

D () obtains reliable disparity map by the parallax value weighted average of different scale:

$d(x,y)=\frac{{\mathrm{\Σ}}_{scale}\[d_{scale}(x,y)\·{\mathrm{\γ}}_{2,scale}(x,y)\]}{{\mathrm{\Σ}}_{scale}{\mathrm{\γ}}_{2,scale}(x,y)}---\left(13\right).$

6., as claimed in claim 5 based on the adaptive three-dimensional parallax estimation method of Bidimensional Empirical Mode Decomposition, it is characterized in that, in described step 5, the acquisition process of final parallax is further comprising the steps of:

The reprocessing of (e) disparity map:

Suppose that neighbor point should have close parallax, the final parallax of each point is the weighted average of itself and neighborhood parallax value, and weight coefficient is determined by the coefficient of reliability of each point.