CN103218771A - Parameter self-adaptation selecting method based on autoregressive model depth recovery - Google Patents

Abstract

The invention discloses a parameter self-adaptation selecting method based on autoregressive model depth recovery, and belongs to the field of computer vision. The parameter self-adaptation selecting method based on the autoregressive model depth recovery aims at achieving recovery of an optimal final depth image according to various characteristics of depth images and color images. According to the technical scheme, the parameter self-adaptation selecting method based on the autoregressive model depth recovery comprises the following steps: (1) specifically formulating a depth recovery problem to be an autoregressive model solving equation, (2) carrying out self-adaptation dereferencing on sigma, namely carrying out the self-adaptation dereferencing on the sigma through texture structures of the color images and an initial depth image, (3) carrying out self-adaptation dereferencing on tg, namely carrying out the self-adaptation dereferencing on the tg through gradient gc of the color images, finally calculating a final prediction coefficient a, and (4) substituting the prediction coefficient a to an energy equation E(f) for final solution. The parameter self-adaptation selecting method based on the autoregressive model depth recovery is mainly applied in self-adaptation selection of multiple parameters in a depth image recovery process.

Description

Parameter adaptive choosing method based on the autoregressive model depth recovery

Technical field

The invention belongs to computer vision field, relate to the self-adaptation choosing method of a plurality of parameters in the depth map recuperation.Specifically, adopt autoregressive model, recovering contaminated (low resolution, the band noise, have random error and structural mistake) in the process of depth map, make the method that each parameter that relates to rejuvenation is chosen according to the picture characteristics self-adaptation, promptly based on the parameter adaptive choosing method of autoregressive model depth recovery.

Background technology

Along with the progress of imaging technique, conventional laser scanning broken through by the degree of depth camera that appears on the market in recent years and three-dimensional coupling is carried out the restriction of Depth Imaging, can obtain the degree of depth of Real-time and Dynamic three-dimensional scenic more easily.But, be subjected to the restriction of many conditions, for example, need to gather in real time, need to realize complete dynamic scene, and the interference that prevents bias light, depth map that degree of depth camera is taken out exists a lot of not enough: 1) resolution low 2) image is subjected to noise pollution serious 3) fathom influenced by bias light and inaccurate 4) exist at random or structurized depth value disappearance.We call the deficiency of these existence the depth map that contains degradation model.Because the existence of these degradation models makes that the application of degree of depth camera reality is not very extensive.So the method for seeking a kind of depth recovery is necessary.

Present stage, work both at home and abroad adopts the high-resolution cromogram through alignment to obtain high-quality depth map with the depth map that contains degradation model.Like this, edge that just in time can corresponding coloured image in the hypothesis degree of depth discontinuous place, and the zone of solid colour is contained under the situation of similar 3D (three-dimension) geometry, can utilize high-resolution chromatic information that the low resolution depth map is carried out super-resolution optimization.The topmost method of one class is to adopt wave filter to optimize depth map.These class methods can be handled low resolution simultaneously, have noise and contain the multiple mixing degradation model that depth value lacks.Wherein, a kind of is to utilize two-sided filter to realize super-resolution; Not by fuzzy,, be applied to depth map because two-sided filter can well keep the edge in filtering, can when enlarging resolution, carry out denoising image according to these characteristics.Recently, also have a kind of method to adopt non-local mean filtering to realize depth recovery.They can judge the characteristic of analog structure by non-local mean filtering, realize super-resolution and image denoising.

But, related to many in the said method and complicated parameter, and all be rule of thumb to be worth to set, parameter is not carried out self-adaptation and choose at the structure of image and characteristic.Different picture depths is recovered only to adopt same kind of parameter setting, will inevitably cause some image well not recover, and perhaps can not reach optimum solution.

Summary of the invention

The present invention is intended to overcome the deficiencies in the prior art, realization recovers optimum ultimate depth figure according to the characteristic of different depth maps and cromogram, and for this reason, the technical scheme that the present invention takes is, parameter adaptive choosing method based on the autoregressive model depth recovery comprises the following steps:

1) the depth recovery problem is expressed as the autoregressive model solving equation particularly:

Equation comprises two parts, wherein, and E _ARBe the autoregressive model item, D is the ultimate depth figure that wishes to get, and D (f), D (p) are respectively the depth value of 2 of f, p, | f-p| represents the space length between neighborhood territory pixel and the current pixel, and unit is a pixel, and S is the current pixel neighborhood of a point, and a is a predictive coefficient; E _DataBe data item, G is the depth information that contains degradation model that can be observed, and G (f) is the depth value of pixel f on G, and ζ is the set of significant depth information among the G, and E (f) is an energy function, and λ is a weighting factor, the weight between two of the balances;

Predictive coefficient a predicts that by known cromogram and the initial depth figure that obtains with the neighbor interpolation method constructive method is as follows: a=a _c* a _g, wherein, a _cBe the coefficient that prediction obtains on cromogram, a _gBe the coefficient that prediction obtains on initial depth figure, equation is as follows:

$a_c=\exp (-\frac{B_f|P\left(I_f\right)-P\left(I_p\right)|}{{2\mathrm{\σ}}^2})$

Wherein, $B_f(f,p)=\exp (-\frac{{|f-p|}^2}{{2\mathrm{\τ}}_1^2})\exp (-\frac{{|I\left(f\right)-I\left(p\right)|}^2}{{2\mathrm{\τ}}_2^2})---\left(2\right)$

$a_g=\exp (-\frac{{(D_g\left(f\right)-D_g\left(p\right))}^2}{{2\mathrm{\τ}}_g^2})---\left(3\right)$

Wherein, for each p point in the neighborhood of current pixel f, P (I _f), P (I _p) be respectively with value of color I _f, I _pPrediction piece for the center; σ is control a _{F, p}The parameter of size, B _fBe the bilateral filtering item, B _f(f, p) representative is from 2 B that calculate of f, p _fIn weights, τ ₁, τ ₂Be respectively the regulation and control parameter of spatial resolution and colour resolution, I (f), I (p) represent the value of color of 2 of f on the cromogram, p; D _gBe the fuzzy initial depth figure after the interpolation, D _g(f), D _g(p) be respectively the depth value of 2 of f, p, τ _gBe control a _gThe parameter of size;

In the above-mentioned equation, following parameter: τ is arranged _g, σ, τ ₁, τ ₂, λ, wherein, τ ₁, τ ₂, λ is different and be consistent along with image restored, is taken as definite value; τ _g, σ adopts different self-adaptation means to come value respectively, method sees 2), 3);

2) σ is carried out the self-adaptation value: texture structure and initial depth figure by cromogram carry out the self-adaptation value to σ;

3) to τ _gCarry out the self-adaptation value: by the gradient g of cromogram _cCome τ _gCarry out the self-adaptation value, last, calculate final predictive coefficient a;

4) predictive coefficient a substitution energy equation E (f) is finally found the solution.

To contain following degradation model respectively: 1) low resolution; 2) low resolution and contain much noise; 3) have random error; 4) depth map that has structural mistake can be tried to achieve final optimization D as a result as the observation depth map G substitution energy equation of needs optimization.

21) utilize the Sigmoid function that the relation between initial depth figure gradient and the σ value is carried out match:

When the pixel that will recover is in degree of depth smooth domain, adopt a big σ value to remove the depth structure that recovers current, promptly the σ value is big more, and weight is more little, and cromogram predicts that partly weight is more little; On the contrary, if pixel is in abundant depth texture zone, adopt a little σ value to come predetermined depth texture information better, the gradient information of initial depth figure can well reflect the degree of enriching of depth texture, so the gradient of employing initial depth figure goes to estimate the value of σ;

22) utilize the span realization self-adaptation of piecewise function to σ

The σ value that on behalf of us, the minimum value that the Sigmoid function can be got can get to peaked scope, according to equation (2), the predictive coefficient a of coloured image _cBe to be that the colored piece P at center predicts according to current pixel, so, the span of σ of having enriched degree direct influence of the texture structure of colored piece comes the span of σ is carried out self-adaptation according to the Grad structural segmentation function of the color texture structure of current pixel piece.

31) pass through g _cCome τ _gCarry out the self-adaptation value;

32) pass through two parameter τ _g, the self-adaptation of σ solves two predictive coefficient a _c, a _g, getting predictive coefficient to the end, equation is as follows:

a＝a _c×a _g （4）

Wherein, a is final predictive coefficient.

Technical characterstic of the present invention and effect:

The inventive method is at the various depth maps that contain degradation model, adopt the depth recovery framework of autoregressive model, by necessary parameter in the rejuvenation is carried out the self-adaptation value, realized recovering optimum ultimate depth figure according to the characteristic of different depth maps and cromogram.Have following characteristics:

1, program is simple, is easy to realize.

2, adopt optimum at present autoregressive model framework that the depth map that contains degradation model is carried out depth recovery.

3, distinguish main influence parameter and the non-main parameter that influences.According to test, the non-main parameter that influences is adopted fixed value, the main parameter that influences is then adopted adaptive approach.Make that like this program execution speed is faster.

4,, not only realized value, and realized dynamic value the σ span according to picture structure for parameter σ.

5, for parameter τ _g, realized that employing color texture structure carries out the self-adaptation value to it.

6,, realized that at the different depth maps that contains degradation model automatic regulation and control parameter means are all arranged, rather than adopted the same value by the self-adaptation of parameter.Make at each special depth map that like this restoration result of an optimum all can be arranged.

Description of drawings

Fig. 1 is actual implementing procedure figure;

Fig. 2 is fuzzy initial depth figure;

Fig. 3 is a cromogram;

Fig. 4 is the depth map that contains degradation model, and is among the figure, upper left: structural disappearance; Lower-left: lack at random; Upper right: the band noise; Bottom right: low resolution;

Fig. 5 is the gradient map of initial depth figure;

Fig. 6 is the Sigmoid functional arrangement;

Fig. 7 is the gradient information figure of cromogram;

Fig. 8 be four kinds of degradation models finally find the solution depth map.Among the figure, restoration result: upper left: structural disappearance; Lower-left: lack at random; Upper right: the band noise; Bottom right: low resolution.

Embodiment

The present invention adopts autoregressive model that the depth map that contains degradation model is carried out depth recovery, by the cromogram of alignment and the initial fuzzy depth map of being constructed are predicted, come the depth map that contains degradation model (band noise, low resolution contain at random, structure disappearance) is recovered; In this process, at the structure of image and characteristic parameter is carried out self-adaptation and choose.Comprise the following steps:

Below in conjunction with embodiment and accompanying drawing the parameter adaptive that the present invention is based on the recovery of autoregressive model frame depth is made a detailed description.

The present invention adopts autoregressive model that the depth map that contains degradation model is carried out depth recovery, and utilize the onesize depth map that provides in the Middlebury data set and cromogram as experimental data, and the hypothesis degree of depth discontinuous place just in time can corresponding coloured image the edge, and similar 3D (three-dimension) geometry is contained in the zone of solid colour.

1) structure primary data and solving equation, and list the adaptive parameter of needs.

11) with MTD figure down-sampling, the mode with the neighbor interpolation method is upsampled to original resolution again, obtains fuzzy initial depth figure.We predict the depth map to containing degradation model (band noise, low resolution contain at random, structure disappearance) to recover by alignment cromogram and initial depth figure.

12) the depth recovery problem is expressed as the autoregressive model solving equation particularly:

Equation comprises two parts, wherein, and E _ARBe the autoregressive model item, D is the depth map of wishing to get, and D (f), D (p) are respectively the depth value of 2 of f, p, | f-p| represents the space length between a f and the some p, and unit is a pixel, and S is the current pixel neighborhood of a point, and a is a predictive coefficient; E _DataBe data item, G is the depth information that contains degradation model that can be observed, and G (f) is the depth value of pixel f on G, and ζ is the set of significant depth information among the G.E (f) is an energy function, and λ is a weighting factor, the weight between two of the balances.

13) predictive coefficient a predicts by known cromogram and the initial depth figure that obtains with the neighbor interpolation method.Constructive method is as follows: a=a _c* a _gWherein, a _cBe the coefficient that prediction obtains on cromogram, a _gBe the coefficient that prediction obtains on initial depth figure:

$a_c=\exp (-\frac{B_f|P\left(I_f\right)-P\left(I_p\right)|}{{2\mathrm{\σ}}^2})$

Wherein, $B_f(f,p)=\exp (-\frac{{|f-p|}^2}{{2\mathrm{\τ}}_1^2})\exp (-\frac{{|I\left(f\right)-I\left(p\right)|}^2}{{2\mathrm{\τ}}_2^2})---\left(2\right)$

$a_g=\exp (-\frac{{(D_g\left(f\right)-D_g\left(p\right))}^2}{{2\mathrm{\τ}}_g^2})---\left(3\right)$

Wherein, for each p point in the neighborhood of current pixel f, P (I _f), P (I _p) be respectively with value of color I _f, I _pPrediction piece for the center; σ is control a _{F, p}The parameter of size, B _fBe the bilateral filtering item of the gaussian kernel that has replaced the local filtering of non-average, B _f(f, p) representative is from 2 B that calculate of f, p _fIn weights, τ ₁, τ ₂Be respectively the regulation and control parameter of spatial resolution and colour resolution, I (f), I (p) represent the value of color of 2 of f on the cromogram, p; a _{F, p}Value big more, illustrate that the similarity of f, two points of p is high more, go to predict that with the p point the accurate probability of f point is very higher.D _gBe the fuzzy initial depth figure after the interpolation, D _g(f), D _g(p) be respectively the depth value of 2 of f, p, τ _gBe control a _gThe parameter of size.

14) in the above-mentioned equation, following parameter: τ is arranged _g, σ, τ ₁, τ ₂, λ.Wherein, τ _gBe controlled at the weight of predicting on the initial depth figure; The weight of predicting on the σ control cromogram; τ ₁The size of control spatial resolution, τ ₂The reversibility of control solving equation; Balance between λ balance data item and the autoregression item.By checking, τ ₁, τ ₂, λ is different and be consistent along with image restored, can be taken as definite value; τ _g, σ adopts different self-adaptation means to come value respectively, method sees 2), 3).

2) σ is carried out the self-adaptation value.

The weight of predicting on the σ control cromogram, and whether smoothly have very big related with actual grade zone that the pixel that will predict and its neighborhood are positioned.So we carry out the self-adaptation value by the texture structure and the initial depth figure of cromogram to σ.

21) utilize the Sigmoid function that the relation between initial depth figure gradient and the σ value is carried out match:

When the pixel that will recover is in degree of depth smooth domain, we adopt a big σ value (be that the σ value is big more, weight is more little, and cromogram predicts that partly weight is more little) to remove the depth structure that recovers current; On the contrary, if pixel is in abundant depth texture zone, we adopt a little σ value to come predetermined depth texture information better.The gradient information of initial depth figure can well reflect the degree of enriching of depth texture, so we adopt the gradient of initial depth figure to go to estimate the value of σ.The gradient method for solving of initial depth figure is as follows:

$g_m=\sqrt{{\left({\mathrm{grad}}_x{D}_g\right)}^2+{\left({\mathrm{grad}}_y{D}_g\right)}^2}---\left(4\right)$

Wherein, D _gBe initial depth figure, grad represents image D _gAsk difference, grad _xBe to ask difference in the direction of horizontal ordinate, grad _yBe to ask difference in the direction of ordinate.

In view of the value of Grad and σ presents inverse relation, we adopt the Sigmoid function to go the relation of match between them, and equation is as follows:

$\mathrm{\σ}=\frac{\max -\min }{1+\exp (-g_m+\mathrm{Phase})}+\min ---\left(5\right)$

Wherein, g _mBe the initial depth Grad of pixel, σ is according to g _mValue and value; The minimum value that on behalf of σ, min can get, the maximal value that on behalf of σ, max can get, max-min represents the span of σ; Phase control matched curve translational movement.

We extract the substantive test image at random out from Middlebury data set the inside and test, because depth information all is that non-ordinary light is sliding at most of location of pixels, most pixel gradient value is all concentrated and is distributed in a scope, and Phase is representing the Sigmoid average value of a function.Therefore, getting Phase is a definite value.

22) utilize the span realization self-adaptation of piecewise function to σ

The σ value that on behalf of us, the minimum value that the Sigmoid function can be got can get to peaked scope.According to equation (2), the predictive coefficient a of coloured image _cBe to be that the colored piece P at center predicts according to current pixel, so, the span of σ of having enriched degree direct influence of the texture structure of colored piece.The gradient information of cromogram can well reflect the degree of enriching of colored piece texture, so we adopt the gradient of cromogram to go to estimate the span of σ.The gradient method for solving of cromogram is as follows:

$g_c=\frac{\sqrt{{\mathrm{\Σ}}_{k\∈\mathrm{\ζ}}({\left({\mathrm{grad}}_x{I}^k\right)}^2+{\left({\mathrm{grad}}_y{I}^k\right)}^2)}}{3}---\left(6\right)$

Wherein, g _cBe the Grad of the colored piece of current pixel, { B}, RGB represent redness respectively for R, G, and be green and blue, I in ζ representative color value space ^kBe the pixel value of current color passage, k takes from the color space under the ζ.

Therefore, for each piece to be asked,, have the span of different σ according to its color texture structure.We fix other parameters σ are tested, and extract the colored piece of substantive test image once more from Middlebury data set the inside at random out and test, and obtain following result:

$\mathrm{\σ}=\left\{\begin{array}{cc}\min =0.35,\max =0.50& g_c\≥8\\ \min =0.25,\max =0.40& 5\≤g_c\≤8\\ \min =0.10,\max =0.30& g_c\≤5\end{array}\right.---\left(7\right)$

Wherein, the minimum value that on behalf of σ, min can get, the maximal value that on behalf of σ, max can get.So far, we have finished the adaptive prediction to parameter σ.

3) to τ _gCarry out the self-adaptation value.

31) though supposed in the discontinuous place of the degree of depth can corresponding coloured image the edge, and similar 3D (three-dimension) geometry is contained in the zone of solid colour, the texture of cromogram is more a lot of than the texture-rich of depth map.Predictive coefficient a _gIn originally level and smooth depth areas the texture structure similar to cromogram appears in order to prevent exactly, and τ _gThen control the weight of predicting on the initial depth figure.τ _gValue too small (being that weight becomes greatly) can cause image restored edge fog (the initial depth figure that is promptly blured influences); τ _gValue is excessive, and then the depth information of Hui Fuing can have the color texture that originally should not have.Because the gradient g of cromogram _cThe texture-rich degree that can reflect the current pixel piece is so pass through g _cCome τ _gCarry out the self-adaptation value, equation is as follows:

${\mathrm{\&tau;}}_g=\left\{\begin{array}{cc}2.5& g_c\&GreaterEqual;8\\ 3& 5\&le;g_c<8\\ 4& g_c<5\end{array}\right.---\left(8\right)$

So far, realized that all need adaptive parameter.

32) pass through two parameter τ _g, the self-adaptation of σ solves two predictive coefficient a _ca _g, getting predictive coefficient to the end, equation is as follows:

a＝a _c×a _g （9）

Wherein, a is final predictive coefficient.

4) predictive coefficient a substitution energy equation E (f) is finally found the solution.

We will contain following degradation model respectively: 1) low resolution; 2) low resolution and contain much noise; 3) have random error; 4) depth map that has structural mistake can be tried to achieve final optimization D as a result as the observation depth map G substitution energy equation of needs optimization.

The present invention adopts autoregressive model that the depth map that contains degradation model is carried out depth recovery (shown in the flow process of Fig. 1), and utilize the onesize depth map that provides in the Middlebury data set and cromogram as experimental data, and the hypothesis degree of depth discontinuous place just in time can corresponding coloured image the edge, and similar 3D (three-dimension) geometry is contained in the zone of solid colour, reaches embodiment in conjunction with the accompanying drawings and is described in detail as follows:

1) structure primary data and solving equation, and list the adaptive parameter of needs.

11) with MTD figure down-sampling, the mode with the neighbor interpolation method is upsampled to original resolution again, obtains fuzzy initial depth figure, as shown in Figure 2.We predict by alignment cromogram (as shown in Figure 3) and initial depth figure, and (band noise, low resolution contain at random, the structure disappearance, and depth map as shown in Figure 4) recovers to containing degradation model.

12) the depth recovery problem is expressed as the autoregressive model solving equation particularly:

Equation comprises two parts, wherein, and E _ARBe the autoregressive model item, D is the depth map of wishing to get, and D (f), D (p) are respectively the depth value of 2 of f, p, and S is the current pixel neighborhood of a point, and a is a predictive coefficient; E _DataBe data item, G is the depth information that contains degradation model that can be observed, and G (f) is the depth value of pixel f on G, and ζ is the set of significant depth information among the G.E (f) is an energy function, and λ is a weighting factor, the weight between two of the balances.

13) predictive coefficient a predicts by known cromogram and the initial depth figure that obtains with the neighbor interpolation method.Constructive method is as follows: a=a _c* a _gWherein, a _cBe the coefficient that prediction obtains on cromogram, a _gBe the coefficient that prediction obtains on initial depth figure:

$a_c=\exp (-\frac{B_f|P\left(I_f\right)-P\left(I_p\right)|}{{2\mathrm{\&sigma;}}^2})$

Wherein, $B_f(f,p)=\exp (-\frac{{|f-p|}^2}{{2\mathrm{\&tau;}}_1^2})\exp (-\frac{{|I\left(f\right)-I\left(p\right)|}^2}{{2\mathrm{\&tau;}}_2^2})---\left(2\right)$

$a_g=\exp (-\frac{{(D_g\left(f\right)-D_g\left(p\right))}^2}{{2\mathrm{\&tau;}}_g^2})---\left(3\right)$

Wherein, for each p point in the neighborhood of current pixel f, P (I _f), P (I _p) be respectively with value of color I _f, I _pPrediction piece for the center; σ is control a _{F, p}The parameter of size, B _fBe the bilateral filtering item of the gaussian kernel that has replaced the local filtering of non-average, B _f(f, p) representative is from 2 B that calculate of f, p _fIn weights, τ ₁, τ ₂Be respectively the regulation and control parameter of spatial resolution and colour resolution, I (f), I (p) represent the value of color of 2 of f on the cromogram, p; a _{F, p}Value big more, illustrate that the similarity of f, two points of p is high more, go to predict that with the p point the accurate probability of f point is very higher.D _gBe the fuzzy depth map after the interpolation, D _g(f), D _g(p) be respectively the depth value of 2 of f, p, τ _gBe control a _gThe parameter of size.

14) in the above-mentioned equation, following parameter: τ is arranged _g, σ, τ ₁, τ ₂, λ.Wherein, τ _gBe controlled at the weight of predicting on the initial depth figure; The weight of predicting on the σ control cromogram; τ ₁The size of control spatial resolution, τ ₂The reversibility of control solving equation; Balance between λ balance data item and the autoregression item.By checking, τ ₁, τ ₂, λ is different and be consistent along with image restored, can be taken as definite value, we set τ here ₁=3, τ ₂=0.1, λ=0.01; τ _g, σ adopts different self-adaptation means to come value respectively, method sees 2), 3).

2) σ is carried out the self-adaptation value

The weight of predicting on the σ control cromogram, and whether smoothly have very big related with actual grade zone that the pixel that will predict and its neighborhood are positioned.So we carry out the self-adaptation value by the texture structure and the initial depth figure of cromogram to σ.

21) utilize the Sigmoid function that the relation between initial depth figure gradient and the σ value is carried out match:

When the pixel that will recover is in degree of depth smooth domain, we adopt a big σ value (be that the σ value is big more, weight is more little, and cromogram predicts that partly weight is more little) to remove the depth structure that recovers current; On the contrary, if pixel is in abundant depth texture zone, we adopt a little σ value to come predetermined depth texture information better.The gradient information of initial depth figure (as shown in Figure 5) can well reflect the degree of enriching of depth texture, so we adopt the gradient of initial depth figure to go to estimate the value of σ.The gradient method for solving of initial depth figure is as follows:

$g_m=\sqrt{{\left({\mathrm{grad}}_x{D}_g\right)}^2+{\left({\mathrm{grad}}_y{D}_g\right)}^2}---\left(4\right)$

Wherein, D _gBe initial depth figure, grad represents image D _gAsk difference, grad _xBe to ask difference in the direction of horizontal ordinate, grad _yBe to ask difference in the direction of ordinate.

In view of the value of Grad and σ presents inverse relation, we adopt the Sigmoid function to go the relation of match between them, and equation is as follows:

$\mathrm{\&sigma;}=\frac{\max -\min }{1+\exp (-g_m+\mathrm{Phase})}+\min ---\left(5\right)$

Wherein, g _mBe the initial depth Grad of pixel, σ is according to g _mValue and value; The minimum value that on behalf of σ, min can get, the maximal value that on behalf of σ, max can get, max-min represents the span of σ; Phase control matched curve translational movement.The image of function as shown in Figure 6.

We extract 100 test pattern pieces out at random from Middlebury data set the inside, and construct initial depth figure, and ask for the gradient of each pixel on the initial depth figure, and the histogram that draws.Can draw according to histogrammic distribution, because depth information all is that non-ordinary light is sliding in most of pixel, most pixel gradient value all is distributed in below 1.5, and therefore, according to test, we are decided to be 1.5. with Phase

22) utilize the span realization self-adaptation of piecewise function to σ

The σ value that on behalf of us, the minimum value that the Sigmoid function can be got can get to peaked scope.According to equation (2), the predictive coefficient a of coloured image _cBe to be that the colored piece P at center predicts according to current pixel, so, the span of σ of having enriched degree direct influence of the texture structure of colored piece.The gradient information of cromogram (as shown in Figure 7) can well reflect the degree of enriching of colored piece texture, so we adopt the gradient of cromogram to go to estimate the span of σ.The gradient method for solving of cromogram is as follows:

$g_c=\frac{\sqrt{{\mathrm{\&Sigma;}}_{k\&Element;\mathrm{\&zeta;}}({\left({\mathrm{grad}}_x{I}^k\right)}^2+{\left({\mathrm{grad}}_y{I}^k\right)}^2)}}{3}---\left(6\right)$

Wherein, g _cBe the Grad of the colored piece of current pixel, { B}, RGB represent redness respectively for R, G, and be green and blue, I in ζ representative color value space ^kBe the pixel value of current color passage, k takes from the color space under the ζ.

Therefore, for each piece to be asked,, have the span of different σ according to its color texture structure.So we fix other parameters σ is tested, extract the colored piece of 100 test patterns once more at random out from Middlebury data set the inside.By observation to the color gradient histogram distribution, Grad is distributed in 3 zones, the color gradient value that is most of piece is distributed between 5 to 8, so the small part point concentrates on less than 5 or greater than 8., we construct a discontinuous point decides σ at 5 and 8 piecewise functions value:

$\mathrm{\&sigma;}=\left\{\begin{array}{cc}\min =0.35,\max =0.50& g_c\&GreaterEqual;8\\ \min =0.25,\max =0.40& 5\&le;g_c\&le;8\\ \min =0.10,\max =0.30& g_c\&le;5\end{array}\right.---\left(7\right)$

Wherein, g _cBe the Grad of the colored piece of current pixel, the minimum value that on behalf of σ, min can get, the maximal value that on behalf of σ, max can get.So far, we have finished the parameter adaptive prediction on cromogram.

3) to τ _gCarry out the self-adaptation value

31) though supposed in the discontinuous place of the degree of depth can corresponding coloured image the edge, and similar 3D (three-dimension) geometry is contained in the zone of solid colour, the texture of cromogram is more a lot of than the texture-rich of depth map.Predictive coefficient a _gIn originally level and smooth depth areas the texture structure similar to cromogram appears in order to prevent exactly, and τ _gThen control the weight of predicting on the initial depth figure.τ _gValue too small (being that weight becomes greatly) can cause image restored edge fog (the initial depth figure that is promptly blured influences); τ _gValue is excessive, and then the depth information of Hui Fuing can have the color texture that originally should not have.Because the gradient g of cromogram _cThe texture-rich degree that can reflect the current pixel piece is so pass through g _cCome τ _gCarry out the self-adaptation value, equation is as follows:

${\mathrm{\&tau;}}_g=\left\{\begin{array}{cc}2.5& g_c\&GreaterEqual;8\\ 3& 5\&le;g_c<8\\ 4& g_c<5\end{array}\right.---\left(8\right)$

Wherein, work as g _cGreater than 8 o'clock, adopt a less τ _g=2.5 limit the unnecessary texture structure of generation; Work as g _cLess than 5 o'clock, adopt a bigger τ _g=4; Work as g _cIn the time of between being in 5 and 8, just get an intermediate value τ _g=3.

So far, realized that all need adaptive parameter.

32) pass through two parameter τ _g, the self-adaptation of σ solves two predictive coefficient a _ca _g, getting predictive coefficient to the end, equation is as follows:

a＝a _c×a _g （9）

Wherein, a is final predictive coefficient.

4) predictive coefficient a substitution energy equation E (f) is finally found the solution.

We will contain following degradation model respectively: 1) low resolution; 2) low resolution and contain much noise; 3) have random error; 4) depth map that has structural mistake can be tried to achieve final optimization D as a result as the observation depth map G substitution energy equation of needs optimization.Four kinds of degradation models finally find the solution depth map as shown in Figure 8.

Claims (3)

1. the parameter adaptive choosing method based on the autoregressive model depth recovery is characterized in that, comprises the following steps:

1) the depth recovery problem is expressed as the autoregressive model solving equation particularly:

Equation comprises two parts, wherein, and E _ARBe the autoregressive model item, D is the ultimate depth figure that wishes to get, and D (f), D (p) are respectively the depth value of 2 of f, p, | f-p| represents the space length between neighborhood territory pixel and the current pixel, and S is the current pixel neighborhood of a point, and a is a predictive coefficient; E _DataBe data item, G is the depth information that contains degradation model that can be observed, and G (f) is the depth value of pixel f on G, and ζ is the set of significant depth information among the G, and E (f) is an energy function, and λ is a weighting factor, the weight between two of the balances;

Predictive coefficient a predicts that by known cromogram and the initial depth figure that obtains with the neighbor interpolation method constructive method is as follows: a=a _c* a _g, wherein, a _cBe the coefficient that prediction obtains on cromogram, a _gBe the coefficient that prediction obtains on initial depth figure, equation is as follows:

$a_c=\exp (-\frac{B_f|P\left(I_f\right)-P\left(I_p\right)|}{{2\mathrm{\&sigma;}}^2})$

Wherein, $B_f(f,p)=\exp (-\frac{{|f-p|}^2}{{2\mathrm{\&tau;}}_1^2})\exp (-\frac{{|I\left(f\right)-I\left(p\right)|}^2}{{2\mathrm{\&tau;}}_2^2})$

（2）

$a_g=\exp (-\frac{{(D_g\left(f\right)-D_g\left(p\right))}^2}{{2\mathrm{\&tau;}}_g^2})---\left(3\right)$

Wherein, for each p point in the neighborhood of current pixel f, P (I _f), P (I _p) be respectively with value of color I _f, I _pPrediction piece for the center; σ is control a _{F, p}The parameter of size, B _fBe the bilateral filtering item, B _f(f, p) representative is from 2 B that calculate of f, p _fIn weights, τ ₁, τ ₂Be respectively the regulation and control parameter of spatial resolution and colour resolution, I (f), I (p) represent the value of color of 2 of f on the cromogram, p; D _gBe the fuzzy initial depth figure after the interpolation, D _g(f), D _g(p) be respectively the depth value of 2 of f, p, τ _gBe control a _gThe parameter of size;

In the above-mentioned equation, following parameter: τ is arranged _g, σ, τ ₁, τ ₂, λ, wherein, τ ₁, τ ₂, λ is different and be consistent along with image restored, is taken as definite value; τ _g, σ adopts different self-adaptation means to come value respectively, method sees 2), 3);

2) σ is carried out the self-adaptation value: texture structure and initial depth figure by cromogram carry out the self-adaptation value to σ;

3) to τ _gCarry out the self-adaptation value: by the gradient g of cromogram _cCome τ _gCarry out the self-adaptation value, last, calculate final predictive coefficient a;

4) predictive coefficient a substitution energy equation E (f) is finally found the solution.

To contain following degradation model respectively: 1) low resolution; 2) low resolution and contain much noise; 3) have random error; 4) depth map that has structural mistake can be tried to achieve final optimization D as a result as the observation depth map G substitution energy equation of needs optimization.

2. the parameter adaptive choosing method based on the autoregressive model depth recovery as claimed in claim 1 is characterized in that, σ is carried out the self-adaptation value be specially:

When the pixel that will recover is in degree of depth smooth domain, adopt a big σ value to remove the depth structure that recovers current, promptly the σ value is big more, and weight is more little, and cromogram predicts that partly weight is more little; On the contrary, if pixel is in abundant depth texture zone, adopt a little σ value to come predetermined depth texture information better, the gradient information of initial depth figure can well reflect the degree of enriching of depth texture, so the gradient of employing initial depth figure goes to estimate the value of σ;

22) utilize the span realization self-adaptation of piecewise function to σ

The σ value that on behalf of us, the minimum value that the Sigmoid function can be got can get to peaked scope, according to equation (2), the predictive coefficient a of coloured image _cBe to be that the colored piece P at center predicts according to current pixel, so, the span of σ of having enriched degree direct influence of the texture structure of colored piece comes the span of σ is carried out self-adaptation according to the Grad structural segmentation function of the color texture structure of current pixel piece.

3. the parameter adaptive choosing method based on the autoregressive model depth recovery as claimed in claim 1 is characterized in that, to τ _gCarrying out the self-adaptation value is specially

31) pass through g _cCome τ _gCarry out the self-adaptation value;

32) pass through two parameter τ _g, the self-adaptation of σ solves two predictive coefficient a _c, a _g, getting predictive coefficient to the end, equation is as follows:

a＝a _c×a _g （4）

Wherein, a is final predictive coefficient.

Images