CN103218771B - Based on the parameter adaptive choosing method of autoregressive model depth recovery - Google Patents

Abstract

The invention belongs to computer vision field, for realizing recovering optimum ultimate depth figure according to the characteristic of different depth maps and cromogram, for this reason, the technical scheme that the present invention takes is, based on the parameter adaptive choosing method of autoregressive model depth recovery, comprise the following steps: 1) depth recovery problem is expressed as autoregressive model solving equation particularly; 2) self-adaptation value is carried out to σ: by the texture structure of cromogram and initial depth figure, self-adaptation value is carried out to σ; 3) to τ _gcarry out self-adaptation value: by the gradient g of cromogram _ccome τ _gcarry out self-adaptation value, finally, calculate final predictive coefficient a; 4) predictive coefficient a is substituted into energy equation E (f) finally to solve.The self-adaptation that the present invention is mainly used in multiple parameter in depth map recuperation is chosen.

Description

Based on the parameter adaptive choosing method of autoregressive model depth recovery

Technical field

The invention belongs to computer vision field, relate to the self-adaptation choosing method of multiple parameter in depth map recuperation.Specifically, adopt autoregressive model, in the contaminated (low resolution of recovery, band noise, with random error and structural mistake) in the process of depth map, make the method that each parameter relating to rejuvenation is chosen according to picture characteristics self-adaptation, namely based on the parameter adaptive choosing method of autoregressive model depth recovery.

Background technology

Along with the progress of imaging technique, the depth camera in recent years appeared on the market breaches conventional laser scanning and Stereo matching carries out the restriction of Depth Imaging, can obtain the degree of depth of real-time moving three-dimensional scene more easily.But, be subject to the restriction of all many condition, such as, need Real-time Collection, need to realize complete dynamic scene, and prevent the interference of bias light, the depth camera depth map taking out exists a lot of not enough: 1) resolution is low by 2) image is by noise pollution serious 3) to fathom by bias light affects inaccurate 4) there is random or structurized depth value disappearance.We call the depth map containing degradation model these deficiencies existed.Because the existence of these degradation models, the application of depth camera reality is made to be not very extensive.So a kind of method finding depth recovery is necessary.

Present stage, work both at home and abroad adopts the high-resolution cromogram through alignment to obtain high-quality depth map with the depth map containing degradation model.Like this, just in time can the edge of corresponding coloured image in the hypothesis discontinuous place of the degree of depth, and the region of solid colour is when containing similar 3D (three-dimension) geometry, high-resolution chromatic information can be utilized to carry out super-resolution optimization to low resolution depth map.The topmost method of one class adopts wave filter to optimize depth map.These class methods can process low resolution simultaneously, with noise and the multiple mixing degradation model containing depth value disappearance.Wherein, one utilizes two-sided filter to realize super-resolution; Because two-sided filter can well keep edge and not fuzzy while filtering, according to this feature, be applied to depth map, while expansion resolution, denoising can be carried out to image.Recently, a kind of method is also had to adopt non-local mean filtering to realize depth recovery.They can judge the characteristic of analog structure by non-local mean filtering, realize super-resolution and image denoising.

But, relate to the many and parameter of complexity in said method, and be all set based on experience value, for the structure and characteristics of image, self-adaptation is not carried out to parameter and choose.Different picture depths is recovered only to adopt same optimum configurations, some image will inevitably be caused well not recover, or can not reach optimum solution.

Summary of the invention

The present invention is intended to overcome the deficiencies in the prior art, realize recovering optimum ultimate depth figure according to the characteristic of different depth maps and cromogram, for this reason, the technical scheme that the present invention takes is, based on the parameter adaptive choosing method of autoregressive model depth recovery, comprise the following steps:

1) depth recovery problem is expressed as autoregressive model solving equation particularly:

Equation comprises two parts, wherein, and E _aRfor autoregressive model item, the depth value that D is the ultimate depth figure wished to get, D (f), D (p) is respectively f, p 2, | f-p| represents the space length between neighborhood territory pixel and current pixel, unit is pixel, and S is current pixel neighborhood of a point, and a is predictive coefficient; E _datafor data item, G is the depth information containing degradation model that can be observed, and G (f) is the depth value of pixel f on G, ζ is the set of significant depth information in G, E (f) is energy function, and λ is weighting factor, the weight between balancing two;

Predictive coefficient a predicts by known cromogram with the initial depth figure that neighbor interpolation method obtains, and constructive method is as follows: a=a _c× a _g, wherein, a _con cromogram, predict the coefficient obtained, a _gbe on initial depth figure, predict the coefficient obtained, equation is as follows:

$a_c=\exp (-\frac{B_f|P\left(I_f\right)-P\left(I_p\right)|}{{2\mathrm{\σ}}^2})$

Wherein, $B_f(f,p)=\exp (-\frac{{|f-p|}^2}{{2\mathrm{\τ}}_1^2})\exp (-\frac{{|I\left(f\right)-I\left(p\right)|}^2}{{2\mathrm{\τ}}_2^2})---\left(2\right)$

$a_g=\exp (-\frac{{(D_g\left(f\right)-D_g\left(p\right))}^2}{{2\mathrm{\τ}}_g^2})---\left(3\right)$

Wherein, for each p point in the neighborhood of current pixel f, P (I _f), P (I _p) be with value of color I respectively _f, I _pcentered by prediction block; σ is control a _f,pthe parameter of size, B _fbe bilateral filtering item, B _f(f, p) representative is from f, p 2 B calculated _fin weights, τ ₁, τ ₂be respectively the regulation and control parameter of spatial resolution and colour resolution, I (f), I (p) represent the value of color of f, p 2 on cromogram; D _gthe fuzzy initial depth figure after interpolation, D _g(f), D _gp () is respectively the depth value of f, p 2, τ _gfor control a _gthe parameter of size;

In above-mentioned equation, there is following parameter: τ _g, σ, τ ₁, τ ₂, λ, wherein, τ ₁, τ ₂, λ is different and be consistent along with the image recovered, and is taken as definite value; τ _g, σ adopts different self-adaptation means to carry out value respectively, and method is shown in 2), 3);

2) self-adaptation value is carried out to σ: by the texture structure of cromogram and initial depth figure, self-adaptation value is carried out to σ;

3) to τ _gcarry out self-adaptation value: by the gradient g of cromogram _ccome τ _gcarry out self-adaptation value, finally, calculate final predictive coefficient a;

4) predictive coefficient a is substituted into energy equation E (f) finally to solve.

Respectively will containing following degradation model: 1) low resolution; 2) low resolution is also containing much noise; 3) with random error; 4) depth map with structural mistake can try to achieve final optimum results D as the Observational depth figure G substitution energy equation needing to optimize.

21) Sigmoid function is utilized to carry out matching to the relation between initial depth figure gradient and σ value:

When the pixel that will recover is in degree of depth smooth domain, adopt a large σ value to go to recover current depth structure, namely σ value is larger, and weight is less, and cromogram fractional prediction weight is less; On the contrary, if pixel is in abundant depth texture region, adopt a little σ value to carry out predetermined depth texture information better, what the gradient information of initial depth figure can well reflect depth texture enriches degree, so, adopt the gradient of initial depth figure to go to estimate the value of σ;

22) span of piecewise function to σ is utilized to realize self-adaptation

The minimum value that can get of Sigmoid function represents to the scope of maximal value the σ value that we can get, according to equation (2), and the predictive coefficient a of coloured image _cbe according to current pixel centered by colored block P carry out predicting, so, the span enriching degree direct influence σ of the texture structure of colored block, the Grad structural segmentation function according to the color texture structure of current pixel block carries out self-adaptation to the span of σ.

31) g is passed through _ccome τ _gcarry out self-adaptation value;

32) pass through two parameter τ _g, the self-adaptation of σ, solves two predictive coefficient a _c, a _g, obtain last predictive coefficient, equation is as follows:

a＝a _c×a _g（4）

Wherein, a is final predictive coefficient.

Technical characterstic of the present invention and effect:

The inventive method is for the various depth map containing degradation model, adopt the depth recovery framework of autoregressive model, by carrying out self-adaptation value to parameter necessary in rejuvenation, achieving and recovering optimum ultimate depth figure according to the characteristic of different depth maps and cromogram.There is following characteristics:

1, program is simple, is easy to realize.

2, autoregressive model framework optimum is at present adopted to carry out depth recovery to the depth map containing degradation model.

3, major effect parameter and non-principal affecting parameters is distinguished.According to test, fixed value is adopted to non-principal affecting parameters, then adaptive approach is adopted to major effect parameter.Make program execution speed faster like this.

4, for parameter σ, not only achieve the value according to picture structure, and achieve the dynamic value to σ span.

5, for parameter τ _g, achieve and adopt color texture structure to carry out self-adaptation value to it.

6, by the self-adaptation of parameter, achieve for the different depth maps containing degradation model, have regulation and control parameter means automatically, instead of adopt the same value.Make for each special depth map like this, all can have the restoration result that optimum.

Accompanying drawing explanation

Fig. 1 is actual implementing procedure figure;

Fig. 2 is fuzzy initial depth figure;

Fig. 3 is cromogram;

Fig. 4 is the depth map containing degradation model, in figure, and upper left: structural disappearance; Lower-left: missing at random; Upper right: band noise; Bottom right: low resolution;

Fig. 5 is the gradient map of initial depth figure;

Fig. 6 is Sigmoid functional arrangement;

Fig. 7 is the gradient information figure of cromogram;

Fig. 8 be four kinds of degradation models finally solve depth map.In figure, restoration result: upper left: structural disappearance; Lower-left: missing at random; Upper right: band noise; Bottom right: low resolution.

Embodiment

The present invention adopts autoregressive model to carry out depth recovery to the depth map containing degradation model, by predicting with the initial fuzzy depth map constructed the cromogram of alignment, come to recover the depth map containing degradation model (band noise, low resolution, containing random, structure disappearance); In the process, carry out self-adaptation for the structure and characteristics of image to parameter to choose.Comprise the following steps:

Below in conjunction with embodiment and accompanying drawing, the parameter adaptive that the present invention is based on the recovery of autoregressive model frame depth is described in detail.

The present invention adopts autoregressive model to carry out depth recovery to the depth map containing degradation model, and utilize the onesize depth map and cromogram that provide in Middlebury data set as experimental data, and suppose that the discontinuous place of the degree of depth just in time can the edge of corresponding coloured image, and similar 3D (three-dimension) geometry is contained in the region of solid colour.

1) construct primary data and solving equation, and list the adaptive parameter of needs.

11) by MTD figure down-sampling, then be upsampled to original resolution by the mode of neighbor interpolation method, obtain fuzzy initial depth figure.We carry out prediction by alignment cromogram and initial depth figure and come to recover the depth map containing degradation model (band noise, low resolution, containing random, structure disappearance).

12) depth recovery problem is expressed as autoregressive model solving equation particularly:

Equation comprises two parts, wherein, and E _aRfor autoregressive model item, D is the depth map wished to get, and D (f), D (p) are respectively the depth value of f, p 2, | f-p| represents the space length between a f and some p, unit is pixel, and S is current pixel neighborhood of a point, and a is predictive coefficient; E _datafor data item, G is the depth information containing degradation model that can be observed, and G (f) is the depth value of pixel f on G, and ζ is the set of significant depth information in G.E (f) is energy function, and λ is weighting factor, the weight between balancing two.

13) predictive coefficient a predicts by known cromogram with the initial depth figure that neighbor interpolation method obtains.Constructive method is as follows: a=a _c× a _g.Wherein, a _con cromogram, predict the coefficient obtained, a _gon initial depth figure, predict the coefficient obtained:

$a_c=\exp (-\frac{B_f|P\left(I_f\right)-P\left(I_p\right)|}{{2\mathrm{\σ}}^2})$

Wherein, $B_f(f,p)=\exp (-\frac{{|f-p|}^2}{{2\mathrm{\τ}}_1^2})\exp (-\frac{{|I\left(f\right)-I\left(p\right)|}^2}{{2\mathrm{\τ}}_2^2})---\left(2\right)$

$a_g=\exp (-\frac{{(D_g\left(f\right)-D_g\left(p\right))}^2}{{2\mathrm{\τ}}_g^2})---\left(3\right)$

Wherein, for each p point in the neighborhood of current pixel f, P (I _f), P (I _p) be with value of color I respectively _f, I _pcentered by prediction block; σ is control a _f,pthe parameter of size, B _fbe the bilateral filtering item of the gaussian kernel that instead of non-average part filter, B _f(f, p) representative is from f, p 2 B calculated _fin weights, τ ₁, τ ₂be respectively the regulation and control parameter of spatial resolution and colour resolution, I (f), I (p) represent the value of color of f, p 2 on cromogram; a _f,pvalue larger, illustrate that the similarity of f, p two points is higher, go prediction f point accuracy probability very higher with p point.D _gthe fuzzy initial depth figure after interpolation, D _g(f), D _gp () is respectively the depth value of f, p 2, τ _gfor control a _gthe parameter of size.

14), in above-mentioned equation, following parameter is had: τ _g, σ, τ ₁, τ ₂, λ.Wherein, τ _gcontrol the weight predicted on initial depth figure; σ controls the weight that cromogram is predicted; τ ₁control the size of spatial resolution, τ ₂control the reversibility of solving equation; λ weighs the balance between data item and autoregression item.By checking, τ ₁, τ ₂, λ is different and be consistent along with the image recovered, and can be taken as definite value; τ _g, σ adopts different self-adaptation means to carry out value respectively, and method is shown in 2), 3).

2) self-adaptation value is carried out to σ.

σ controls the weight that cromogram is predicted, and whether smoothly has very large association with the actual grade region at the pixel that will predict and its neighborhood.So we carry out self-adaptation value by the texture structure of cromogram and initial depth figure to σ.

21) Sigmoid function is utilized to carry out matching to the relation between initial depth figure gradient and σ value:

When the pixel that will recover is in degree of depth smooth domain, we adopt a large σ value (namely σ value is larger, and weight is less, and cromogram fractional prediction weight is less) to go to recover current depth structure; On the contrary, if pixel is in abundant depth texture region, we adopt a little σ value to carry out predetermined depth texture information better.What the gradient information of initial depth figure can well reflect depth texture enriches degree, so we adopt the gradient of initial depth figure to go to estimate the value of σ.The gradient method for solving of initial depth figure is as follows:

$g_m=\sqrt{{\left({\mathrm{grad}}_x{D}_g\right)}^2+{\left({\mathrm{grad}}_y{D}_g\right)}^2}---\left(4\right)$

Wherein, D _gfor initial depth figure, grad representative is to image D _gask difference, grad _xask difference, grad in the direction of horizontal ordinate _yask difference in the direction of ordinate.

In view of the value of Grad and σ presents inverse relation, the relation that we adopt Sigmoid function to go matching between them, equation is as follows:

$\mathrm{\σ}=\frac{\max -\min }{1+\exp (-g_m+\mathrm{Phase})}+\min ---\left(5\right)$

Wherein, g _mbe the initial depth Grad of pixel, σ is according to g _mvalue and value; Min represents the minimum value that σ can get, and max represents the maximal value that σ can get, and max-min represents the span of σ; Phase controls matched curve translational movement.

We test by a large amount of test pattern of random extraction inside Middlebury data set, because depth information is all unusual light at most of location of pixels, all integrated distribution is a scope for most pixel gradient value, and Phase represents Sigmoid average value of a function.Therefore, getting Phase is a definite value.

22) span of piecewise function to σ is utilized to realize self-adaptation

The minimum value that can get of Sigmoid function represents to the scope of maximal value the σ value that we can get.According to equation (2), the predictive coefficient a of coloured image _cbe according to current pixel centered by colored block P carry out predicting, so, the span enriching degree direct influence σ of the texture structure of colored block.What the gradient information of cromogram can well reflect colored block texture enriches degree, so we adopt the gradient of cromogram to go to estimate the span of σ.The gradient method for solving of cromogram is as follows:

$g_c=\frac{\sqrt{{\mathrm{\Σ}}_{k\∈\mathrm{\ζ}}({\left({\mathrm{grad}}_x{I}^k\right)}^2+{\left({\mathrm{grad}}_y{I}^k\right)}^2)}}{3}---\left(6\right)$

Wherein, g _cbe the Grad of the colored block of current pixel, { R, G, B}, RGB represent redness to ζ representative color valued space respectively, green and blue, I ^kfor the pixel value of current color passage, k takes from the color space belonging to ζ.

Therefore, for each block to be asked, according to its color texture structure, have the span of different σ.We fix other parameters and test σ, and inside Middlebury data set, the random colored block extracting a large amount of test pattern out is tested again, obtains following result:

$\mathrm{\σ}=\left\{\begin{array}{cc}\min =0.35,\max =0.50& g_c\≥8\\ \min =0.25,\max =0.40& 5\≤g_c\≤8\\ \min =0.10,\max =0.30& g_c\≤5\end{array}\right.---\left(7\right)$

Wherein, min represents the minimum value that σ can get, and max represents the maximal value that σ can get.So far, we have completed to parameter σ adaptive prediction.

3) to τ _gcarry out self-adaptation value.

31) although supposed in the discontinuous place of the degree of depth can the edge of corresponding coloured image, and similar 3D (three-dimension) geometry is contained in the region of solid colour, but the texture of cromogram is more a lot of than the texture-rich of depth map.Predictive coefficient a _gthe texture structure similar to cromogram is there is in originally level and smooth depth areas exactly in order to prevent, and τ _gthen control the weight that initial depth figure predicts.τ _gvalue too small (namely weight becomes large), can cause the soft edge (namely affecting by fuzzy initial depth figure) recovered; τ _gvalue is excessive, then the depth information recovered can with the color texture that originally should not have.Because the gradient g of cromogram _cthe texture-rich degree of current pixel block can be reflected, so pass through g _ccome τ _gcarry out self-adaptation value, equation is as follows:

${\mathrm{\&tau;}}_g=\left\{\begin{array}{cc}2.5& g_c\&GreaterEqual;8\\ 3& 5\&le;g_c<8\\ 4& g_c<5\end{array}\right.---\left(8\right)$

So far, the adaptive parameter of all needs is achieved.

32) pass through two parameter τ _g, the self-adaptation of σ, solves two predictive coefficient a _ca _g, obtain last predictive coefficient, equation is as follows:

a＝a _c×a _g（9）

Wherein, a is final predictive coefficient.

4) predictive coefficient a is substituted into energy equation E (f) finally to solve.

We respectively will containing following degradation model: 1) low resolution; 2) low resolution is also containing much noise; 3) with random error; 4) depth map with structural mistake can try to achieve final optimum results D as the Observational depth figure G substitution energy equation needing to optimize.

The present invention adopts autoregressive model to carry out depth recovery (as shown in the flow process of Fig. 1) to the depth map containing degradation model, and utilize the onesize depth map and cromogram that provide in Middlebury data set as experimental data, and suppose that the discontinuous place of the degree of depth just in time can the edge of corresponding coloured image, and similar 3D (three-dimension) geometry is contained in the region of solid colour, by reference to the accompanying drawings and embodiment be described in detail as follows:

1) construct primary data and solving equation, and list the adaptive parameter of needs.

11) by MTD figure down-sampling, then be upsampled to original resolution by the mode of neighbor interpolation method, obtain fuzzy initial depth figure, as shown in Figure 2.We carry out prediction by alignment cromogram (as shown in Figure 3) and initial depth figure and come (, containing random, structure disappearance, depth map as shown in Figure 4) recovers for band noise, low resolution containing degradation model.

12) depth recovery problem is expressed as autoregressive model solving equation particularly:

Equation comprises two parts, wherein, and E _aRfor autoregressive model item, D is the depth map wished to get, and D (f), D (p) are respectively the depth value of f, p 2, and S is current pixel neighborhood of a point, and a is predictive coefficient; E _datafor data item, G is the depth information containing degradation model that can be observed, and G (f) is the depth value of pixel f on G, and ζ is the set of significant depth information in G.E (f) is energy function, and λ is weighting factor, the weight between balancing two.

13) predictive coefficient a predicts by known cromogram with the initial depth figure that neighbor interpolation method obtains.Constructive method is as follows: a=a _c× a _g.Wherein, a _con cromogram, predict the coefficient obtained, a _gon initial depth figure, predict the coefficient obtained:

$a_c=\exp (-\frac{B_f|P\left(I_f\right)-P\left(I_p\right)|}{{2\mathrm{\&sigma;}}^2})$

Wherein, $B_f(f,p)=\exp (-\frac{{|f-p|}^2}{{2\mathrm{\&tau;}}_1^2})\exp (-\frac{{|I\left(f\right)-I\left(p\right)|}^2}{{2\mathrm{\&tau;}}_2^2})---\left(2\right)$

$a_g=\exp (-\frac{{(D_g\left(f\right)-D_g\left(p\right))}^2}{{2\mathrm{\&tau;}}_g^2})---\left(3\right)$

Wherein, for each p point in the neighborhood of current pixel f, P (I _f), P (I _p) be with value of color I respectively _f, I _pcentered by prediction block; σ is control a _f,pthe parameter of size, B _fbe the bilateral filtering item of the gaussian kernel that instead of non-average part filter, B _f(f, p) representative is from f, p 2 B calculated _fin weights, τ ₁, τ ₂be respectively the regulation and control parameter of spatial resolution and colour resolution, I (f), I (p) represent the value of color of f, p 2 on cromogram; a _f,pvalue larger, illustrate that the similarity of f, p two points is higher, go prediction f point accuracy probability very higher with p point.D _gthe fuzzy depth map after interpolation, D _g(f), D _gp () is respectively the depth value of f, p 2, τ _gfor control a _gthe parameter of size.

14), in above-mentioned equation, following parameter is had: τ _g, σ, τ ₁, τ ₂, λ.Wherein, τ _gcontrol the weight predicted on initial depth figure; σ controls the weight that cromogram is predicted; τ ₁control the size of spatial resolution, τ ₂control the reversibility of solving equation; λ weighs the balance between data item and autoregression item.By checking, τ ₁, τ ₂, λ along with the image recovered different and be consistent, can be taken as definite value, we set τ here ₁=3, τ ₂=0.1, λ=0.01; τ _g, σ adopts different self-adaptation means to carry out value respectively, and method is shown in 2), 3).

2) self-adaptation value is carried out to σ

σ controls the weight that cromogram is predicted, and whether smoothly has very large association with the actual grade region at the pixel that will predict and its neighborhood.So we carry out self-adaptation value by the texture structure of cromogram and initial depth figure to σ.

21) Sigmoid function is utilized to carry out matching to the relation between initial depth figure gradient and σ value:

When the pixel that will recover is in degree of depth smooth domain, we adopt a large σ value (namely σ value is larger, and weight is less, and cromogram fractional prediction weight is less) to go to recover current depth structure; On the contrary, if pixel is in abundant depth texture region, we adopt a little σ value to carry out predetermined depth texture information better.What the gradient information (as shown in Figure 5) of initial depth figure can well reflect depth texture enriches degree, so we adopt the gradient of initial depth figure to go to estimate the value of σ.The gradient method for solving of initial depth figure is as follows:

$g_m=\sqrt{{\left({\mathrm{grad}}_x{D}_g\right)}^2+{\left({\mathrm{grad}}_y{D}_g\right)}^2}---\left(4\right)$

Wherein, D _gfor initial depth figure, grad representative is to image D _gask difference, grad _xask difference, grad in the direction of horizontal ordinate _yask difference in the direction of ordinate.

In view of the value of Grad and σ presents inverse relation, the relation that we adopt Sigmoid function to go matching between them, equation is as follows:

$\mathrm{\&sigma;}=\frac{\max -\min }{1+\exp (-g_m+\mathrm{Phase})}+\min ---\left(5\right)$

Wherein, g _mbe the initial depth Grad of pixel, σ is according to g _mvalue and value; Min represents the minimum value that σ can get, and max represents the maximal value that σ can get, and max-min represents the span of σ; Phase controls matched curve translational movement.The image of function as shown in Figure 6.

We are random inside Middlebury data set extracts 100 test pattern blocks out, constructs initial depth figure, and asks for the gradient of each pixel on initial depth figure, and draw histogram.Can draw according to histogrammic distribution, because depth information is all unusual light in most of pixel, most pixel gradient value is all distributed in less than 1.5, and therefore, according to test, Phase is decided to be 1.5. by us

22) span of piecewise function to σ is utilized to realize self-adaptation

The minimum value that can get of Sigmoid function represents to the scope of maximal value the σ value that we can get.According to equation (2), the predictive coefficient a of coloured image _cbe according to current pixel centered by colored block P carry out predicting, so, the span enriching degree direct influence σ of the texture structure of colored block.What the gradient information (as shown in Figure 7) of cromogram can well reflect colored block texture enriches degree, so we adopt the gradient of cromogram to go to estimate the span of σ.The gradient method for solving of cromogram is as follows:

$g_c=\frac{\sqrt{{\mathrm{\&Sigma;}}_{k\&Element;\mathrm{\&zeta;}}({\left({\mathrm{grad}}_x{I}^k\right)}^2+{\left({\mathrm{grad}}_y{I}^k\right)}^2)}}{3}---\left(6\right)$

Wherein, g _cbe the Grad of the colored block of current pixel, { R, G, B}, RGB represent redness to ζ representative color valued space respectively, green and blue, I ^kfor the pixel value of current color passage, k takes from the color space belonging to ζ.

Therefore, for each block to be asked, according to its color texture structure, have the span of different σ.So we fix other parameters, σ is tested, again the random colored block extracting 100 test patterns out inside Middlebury data set.By the observation to color gradient histogram distribution, Grad is distributed in 3 regions, namely the color gradient Distribution value of most of block is between 5 to 8, small part point concentrate on be less than 5 or be greater than 8. so, we construct a discontinuous point decides σ value at 5 and 8 piecewise functions:

$\mathrm{\&sigma;}=\left\{\begin{array}{cc}\min =0.35,\max =0.50& g_c\&GreaterEqual;8\\ \min =0.25,\max =0.40& 5\&le;g_c\&le;8\\ \min =0.10,\max =0.30& g_c\&le;5\end{array}\right.---\left(7\right)$

Wherein, g _cbe the Grad of the colored block of current pixel, min represents the minimum value that σ can get, and max represents the maximal value that σ can get.So far, we have completed the parameter adaptive prediction on cromogram.

3) to τ _gcarry out self-adaptation value

31) although supposed in the discontinuous place of the degree of depth can the edge of corresponding coloured image, and similar 3D (three-dimension) geometry is contained in the region of solid colour, but the texture of cromogram is more a lot of than the texture-rich of depth map.Predictive coefficient a _gthe texture structure similar to cromogram is there is in originally level and smooth depth areas exactly in order to prevent, and τ _gthen control the weight that initial depth figure predicts.τ _gvalue too small (namely weight becomes large), can cause the soft edge (namely affecting by fuzzy initial depth figure) recovered; τ _gvalue is excessive, then the depth information recovered can with the color texture that originally should not have.Because the gradient g of cromogram _cthe texture-rich degree of current pixel block can be reflected, so pass through g _ccome τ _gcarry out self-adaptation value, equation is as follows:

${\mathrm{\&tau;}}_g=\left\{\begin{array}{cc}2.5& g_c\&GreaterEqual;8\\ 3& 5\&le;g_c<8\\ 4& g_c<5\end{array}\right.---\left(8\right)$

Wherein, g is worked as _cwhen being greater than 8, adopt the τ that less _g=2.5 limit the unnecessary texture structure of generation; Work as g _cwhen being less than 5, adopt the τ that larger _g=4; Work as g _cwhen being between 5 and 8, just get an intermediate value τ _g=3.

So far, the adaptive parameter of all needs is achieved.

32) pass through two parameter τ _g, the self-adaptation of σ, solves two predictive coefficient a _ca _g, obtain last predictive coefficient, equation is as follows:

a＝a _c×a _g（9）

Wherein, a is final predictive coefficient.

4) predictive coefficient a is substituted into energy equation E (f) finally to solve.

We respectively will containing following degradation model: 1) low resolution; 2) low resolution is also containing much noise; 3) with random error; 4) depth map with structural mistake can try to achieve final optimum results D as the Observational depth figure G substitution energy equation needing to optimize.Four kinds of degradation models finally solve depth map as shown in Figure 8.

Claims (1)

1., based on a parameter adaptive choosing method for autoregressive model depth recovery, it is characterized in that, comprise the following steps:

1) depth recovery problem is expressed as autoregressive model solving equation particularly:

Equation comprises two parts, wherein, and E _aRfor autoregressive model item, the depth value that D is the ultimate depth figure wished to get, D (f), D (p) is respectively f, p 2, | f-p| represents the space length between neighborhood territory pixel and current pixel, S is current pixel neighborhood of a point, and a is predictive coefficient; E _datafor data item, G is the depth information containing degradation model that can be observed, and G (f) is the depth value of pixel f on G, be the set of significant depth information in G, E (f) is energy function, and λ is weighting factor, the weight between balancing two;

Predictive coefficient a predicts by known cromogram with the initial depth figure that neighbor interpolation method obtains, and constructive method is as follows: a=a _c× a _g, wherein, a _con cromogram, predict the coefficient obtained, a _gbe on initial depth figure, predict the coefficient obtained, equation is as follows:

$a_c=\exp (-\frac{B_f|P\left(I_f\right)-P\left(I_p\right)|}{2{\mathrm{\&sigma;}}^2})$

Wherein, $B_f(f,p)=\exp (-\frac{{|f-p|}^2}{2{\mathrm{\&tau;}}_1^2})\exp (-\frac{{|I\left(f\right)-I\left(p\right)|}^2}{2{\mathrm{\&tau;}}_2^2})---\left(2\right)$

$a_g=\exp (-\frac{{(D_g\left(f\right)-D_g\left(p\right))}^2}{2{\mathrm{\&tau;}}_g^2})---\left(3\right)$

Wherein, for each p point in the neighborhood of current pixel f, P (I _f), P (I _p) be with value of color I respectively _f, I _pcentered by prediction block; σ is control a _f,pthe parameter of size, B _fbe bilateral filtering item, B _f(f, p) representative is from f, p 2 B calculated _fin weights, τ ₁, τ ₂be respectively the regulation and control parameter of spatial resolution and colour resolution, I (f), I (p) represent the value of color of f, p 2 on cromogram; D _gthe fuzzy initial depth figure after interpolation, D _g(f), D _gp () is respectively the depth value of f, p 2, τ _gfor control a _gthe parameter of size;

In above-mentioned equation, there is following parameter: τ _g, σ, τ ₁, τ ₂, λ, wherein, τ ₁, τ ₂, λ is different and be consistent along with the image recovered, and is taken as definite value; τ _g, σ adopts different self-adaptation means to carry out value respectively, and method is shown in 2), 3);

2) self-adaptation value is carried out to σ: by the texture structure of cromogram and initial depth figure, self-adaptation value is carried out to σ;

3) to τ _gcarry out self-adaptation value: by the gradient g of cromogram _ccome τ _gcarry out self-adaptation value, finally, calculate final predictive coefficient a;

4) predictive coefficient a is substituted into energy equation E (f) finally to solve;

Respectively will containing following degradation model: 1) low resolution; 2) low resolution is also containing much noise; 3) with random error; 4) depth map with structural mistake can try to achieve final optimum results D as the Observational depth figure G substitution energy equation needing to optimize;

21) carry out self-adaptation value to σ to be specially: when the pixel that will recover is in degree of depth smooth domain, adopt a large σ value to go to recover current depth structure, namely σ value is larger, and weight is less, and cromogram fractional prediction weight is less; On the contrary, if pixel is in abundant depth texture region, adopt a little σ value to carry out predetermined depth texture information better, what the gradient information of initial depth figure can well reflect depth texture enriches degree, so, adopt the gradient of initial depth figure to go to estimate the value of σ;

22) span of piecewise function to σ is utilized to realize self-adaptation

The minimum value that can get of Sigmoid function represents to the scope of maximal value the σ value that we can get, according to equation (2), and the predictive coefficient a of coloured image _cbe according to current pixel centered by colored block P carry out predicting, so, the span enriching degree direct influence σ of the texture structure of colored block, the Grad structural segmentation function according to the color texture structure of current pixel block carries out self-adaptation to the span of σ.