CN105513120A - Adaptive rendering method based on weight local regression - Google Patents

Abstract

The invention discloses an adaptive rendering method based on weight local regression. The method comprises the following specific steps: filtering and reconstructing an image by use of local regression; constructing a simplified feature space by use of truncated singular value decomposition (TSVD) so as to eliminate noise carried by a feature vector; respectively predicating an optimal feature bandwidth bj and a shared bandwidth h by use of a two-step bandwidth selection algorithm; and distributing available light samples to areas with high errors by use of an iteration method based on a local simplified feature subspace k. The method provided by the invention is a novel adaptive rendering technology and can be applied to processing rendering effects of multiple types in an efficient and robust manner. Compared to prior similar arts, the method has the following advantages: the time for generating results with the same quality is obviously reduced, and the result image effect generated based on equal calculation time and calculation cost is far better than that of the similar arts.

Description

Based on the Adapti ve rendering method of weighting local regression

Technical field

The present invention relates to computer image processing technology field, is a kind of Adapti ve rendering image reconstructing method returned based on weighting local.

Background technology

Adapti ve rendering method and Monte Carlo ray follow the trail of reconstruction image very long phase of history.Current this field that is operated in also achieves larger progress, but main target remains identical: namely use the light sample of lesser amt to go to improve picture quality to raise the efficiency.The image generated by very few light sample is easily subject to noise and to aggregate into the process of a width smoothed image very slow.The key factor of self-adaptation sample and reconstruct is that error analysis is carried out in local, and this is for instructing light to concentrate on high level error region when image reconstruction controls level and smooth, in order to generation numerically and visually gratifying rendering result.

On high-dimensional, Adapti ve rendering technology can be divided into integration method and image space method.Powerful integration method was suggested as far back as 2008, but the effective image space method of design one is absorbed in recent research always.This is mainly because image space method effectively and be easy to be integrated in existing rendering system, and can process the rendering effect such as motion blur of Various Complex, regional depth etc.

Recent image space reconfiguration technique employs the famous filter method be highlighted in image processing field as Gaussian filter, non local method, associating bilateral filtering, and microwave.The reconfiguration technique that image procossing grows up and those key differences used on playing up are to be applied in the filter method played up and are applicable to utilizing dissimilar available feature, such as normal, texture and the degree of depth.Due to these features noise removing MC rendering result, a lot of existing image space technology is all used to the feature of some types and obtains desirable result.

Unfortunately, these features also can be regarded as double-edged sword.Even feature also can be very noisy, especially when we have compound movement, solid, texture and opticpath scene time.In addition, dissimilar feature has different importance in overall filtering, and this shows as a stray parameter and filters.But these problems are not almost paid attention to existing make use of in the image space Adapti ve rendering method of these features.

Summary of the invention

The present invention proposes a kind of Adapti ve rendering method of self-adapting reconstruction based on local weighted recurrence and Sampling techniques, error analysis can be obtained by the method for a robust, weigh the importance of dissimilar feature, and eliminate the dimensional problem that we consider more multiple features on continuous print fixing means.

Based on an Adapti ve rendering method for weighting local regression, comprising:

Step 1, carries out the feature space of the simplification of truncated singular value decomposition (TSVD) calculating input image for input picture;

Step 2, the method based on local rank of matrix changes the proper vector in input picture global space into simplification vector in the feature space of described simplification;

As mentioned above, the predictive variable of conventional local regression method supposition input is not noisy, and the variable of correspondence has noise, and in order to address this problem, the present invention's use is blocked SVD (TSVD) and constructed a simplification feature space.In the optimization problem solving each pixel, svd (SVD) is used to build as being defined in filter window at one the method that simplifies local feature space.This feature vector, X in global space changes a vectorial z in local space into, implements optimization subsequently in the feature space simplified.

The changes in coordinates that SVD provides reliably can solve the problem that rank defect system can run into usually.When in conjunction with perturbation theory, can cause poor situation or the failed noise reconstructed to reduce those in preconditioning matrix space.When higher level, can think that this process is a preprocessing process, such a process reduces the noise be included in proper vector, and not rely on corresponding vector (such as, density).In addition, coordinate transform can identify new orthogonal characteristic type this is linearly combined with some original characteristic types.

Such as, as a 3D input feature value [x ₁, x ₂, x ₃] when providing, the space of the simplification calculated by SVD can be a 2D vector [x ₁, 0.5x ₂+ 0.5x ₃].

Step 3, builds local regression basic statistical model y=f (x)+∈; Described simplification vector is utilized to obtain input picture f (x) after processing based on the SVD removal noise ∈ that blocks of perturbation theory;

Wherein y represents the input picture with noise ∈; Described noise ∈ is made up of deviation and variance:

Z is described simplification vector;

for the mapping relations of shared bandwidth h, feature bandwidth b and z;

In view of being input as colour picture, the inventive method is applied to each passage independently.Proper vector x ∈ R ^dthere is D to tie up, comprise the position of image and geological information additional arbitrarily, comprise texture, the degree of depth, normal.In order to calculate the proper vector at a pixel place, by the geometric configuration equalization gone out from multiple base light line computation.

In statistical model y=f (x)+∈, suppose that x is a muting R ^dvector, proper vector x can become noisy due to distributed effects such as Depth Domain.This is the committed step applying local regression on playing up to solve this problem by regularization method in the present invention.The unknown central feature vector x of density function f (x) near x ^ccan be general launch with Taylor polynomial, as follows:

$f\left(x\right)\≈f\left(x^c\right)+\▿f{\left(x^c\right)}^T(x-x^c)$

For simplicity, position image value Local Linear Model f (x is made ^c) and its gradient be α and β separately.A weighted least-squares minimizes can determine factor alpha and β, as follows:

$\[\widehat{\mathrm{\α}},\widehat{\mathrm{\β}}\]=\underset{\mathrm{\α},\mathrm{\β}}{min}\underset{i=1}{\overset{n}{\Σ}}{(y^i-\mathrm{\α}-{\mathrm{\β}}^T(x^i-x^c\left)\right)}^2{\Π}_{j=1}^{D}w\left(\frac{x_j^i-x_j^c}{{\mathrm{hb}}_j}\right)$

X and y is proper vector at pixel i place and density herein.

N is pixel index, namely travels through all pixels in formula;

J is a mathematics count number, for the cost in the D of zoning;

H is for sharing bandwidth;

B _jfor feature bandwidth;

In order to filter a central feature vector x ^cthe adjacent feature vector defining it in filter window is x ⁱ.Calculate factor alpha and the β corresponding image that filtered and the prediction gradient to reconstruction separately.Wherein the multidimensional core based on one dimension core product.

In order to prediction deviation and variance more accurately, employ the deviation and variance fallout predictor that develop from existing local regression document.Bias term follows following progressive relation:

In formula:

B is diagonal matrix, is expressed as

Trace () is matrix trace;

it is Hesaian matrix.

In addition,

In formula:

N (z) represents the sample size at z place;

K is the space of each dimension in the feature space simplified;

B _jfor feature bandwidth.Wherein, described feature bandwidth b (i.e. b in following formula _j) be:

B _jfor ${\left|\frac{{\∂}^{2}f}{\∂z_j^2}\right|}^{-0.5}$

F is the mapping relations of b and z;

Z simplifies the vectorial z in local space.

Wherein, described shared bandwidth item h (i.e. h in following formula _opt) determination:

$h_{opt}={\frac{{\mathrm{k\κ}}_1}{4{\mathrm{\λ}}_1^{2}n\left(z\right)}}^{\frac{1}{k+4}}$

K is the space of each dimension in the feature space simplified;

κ ₁for the coefficient using least square error to calculate.

Pass through λ can be determined ₁, λ in this formula ₀and λ ₁it is all the coefficient of prediction.Step 4, each pixel z for the input picture after process arranges the variable quantity of light sample size, then plays up;

The variable quantity of light sample size $\mathrm{\Δ}n\left(z\right)=\frac{\mathrm{\Δ}rMSE\left(z\right)}{{\mathrm{\Σ}}_t\mathrm{\Δ}rMSE\left(z^t\right)};$

this formula is used to calculating variable quantity, and t is a mathematical quantity;

Wherein Δ rMSE (z) is the noise varience of red channel of the input picture after process;

ε is used to avoid denominator to be zero;

$\mathrm{\Δ}MSE\left(z\right)=MSE\left(z\right)\×n{\left(z\right)}^{-\frac{4}{k+4}};$

K is the space of each dimension in the feature space simplified.

Present invention uses a common alternative manner available light sample to be assigned to the region of high level error.As primary iteration, we have been uniformly distributed a small amount of light sample (such as, each pixel four light samples).In ensuing iteration, during by pixel by sample extra for acceptance one, for pixel Z predicts error-reduction item Δ MSE (z).Sample size Δ n (z) is determined subsequently according to the relative value of Δ MSE (z).

Present invention uses the simplification proper subspace k based on local, instead of the error metrics of original feature space D.

Because reconstruct implements the shared bandwidth h calculated _opt, its reconstructed error is predicted to be herein easily find out with h _optdeviation and variance item bias _h(z) and var _h(z).On the other hand, MSE (z) also reduces with identical speed.Δ MSE (z) is defined as the minimizing two of MSE and uses rate of decay to be calculated as $MSE\left(z\right)\×$ $n{\left(z\right)}^{-\frac{4}{k+4}}.$

In order to consider that the visually-perceptible of the mankind employs relative MSE (MesnSquaredError, mean square is poor) to dark portion is more responsive, copy rMSE, then Δ rMSE (z) is defined as follows: ε is used to avoid denominator to be zero herein, is set as 0.001 in actual applications.Subsequently for pixel z arranges sample size Δ n (z), according to its attenuation rate relative to all pixel quantities.In other words, be sampled as pixel z by low contradiction and generate Δ n (z) sample, this is a generally selection used.

The present invention demonstrates one and simplifies local feature space based on blocking SDV (monodrome structure) and perturbation theory and can effectively instructing.

Mean Square Error (MSE) in reconstructing method of the present invention is deconstructed into deviation and variance two kinds and robustly predicts based on parameter error analysis.

The present invention uses partial derivative to predict the importance of each characteristic type and the filter width calculated each feature, creates effective anisotropic filtering.

The dimension that present invention uses from the simplification of local feature spatial analysis carrys out best divergent rays.

Accompanying drawing explanation

Fig. 1 (a) is the schematic diagram of the pending image 128spp (rMSE0.06264) of input;

The pending image of Fig. 1 (b) is through the schematic diagram of the local dimension that use TSVD obtains;

Fig. 1 (c) is reconstruction result when rank of matrix is 2, and filtration time is 4.3 seconds rMSE0.01257;

Fig. 1 (d) is reconstruction result when rank of matrix is 9, and filtration time is 20.6 seconds rMSE0.00666;

Fig. 1 (e) is for using the reconstruction result of the inventive method, and filtration time is 7.2 seconds rMSE0.00488;

Fig. 2 (a) is the pending image (32spp) of input;

Fig. 2 (b) is for predicting that bandwidth is the schematic diagram of hb1;

Fig. 2 (c) is for predicting that bandwidth is the schematic diagram of hb2;

Fig. 2 (d) is for predicting that bandwidth is the schematic diagram of hb3;

Fig. 2 (e) is the result of the wide hbj=0.2 of smaller strip, rMSE0.00886;

Fig. 2 (f) is the result compared with large bandwidth hbj=1, rMSE0.14781;

The result that Fig. 2 (g) is the present embodiment, rMSE0.00176;

Fig. 3 (a) plays up the result of SanMiguel scene, 115spp (660s), rMSE0.00448 for the present invention;

Fig. 3 (b) is respectively from left to right:

LD method plays up the result 128spp (665s) of SanMiguel scene, rMSE0.06288;

NLM method plays up the result 115spp (665s) of SanMiguel scene, rMSE0.01242;

SURE method plays up the result 113spp (665s) of SanMiguel scene, rMSE0.01521;

The 115spp (660s) of the inventive method, rMSE0.00448;

Reference result, 16Kspp;

When Fig. 4 (a) is for computed image marginal position, given noisy image border;

When Fig. 4 (b) is for computed image marginal position, given prediction derives;

When Fig. 4 (c) is for computed image marginal position, given bandwidth;

The result that Fig. 4 (d) obtains for using linear approximation method;

The result that Fig. 4 (e) obtains for the present embodiment.

Embodiment

Below in conjunction with accompanying drawing, the present invention is described in detail with concrete enforcement sample.The present embodiment uses CUDA to realize on the basis of pbrt2 (the physics rendering system based on ray tracing).Accumulate the characteristic sum color buffer in each sample phase and buffer-stored be used in the texture memory of GPU run reconstruct.The CUDA employing a Jacob iteration realizes going to calculate SVD.

The feature space of the simplification of truncated singular value decomposition (TSVD) calculating input image is carried out for input picture.

Provide an input picture, as shown in Fig. 1 (a), calculate local dimension according to TSVD, the results are shown in Figure 1 (b).

If when using the picture of less and larger rank of matrix reconstruct, respectively see Fig. 1 (c) and Fig. 1 (d), each comfortable focal zone has showed excessive fuzzy manual operation and the noise in region out of focus.

And the present embodiment is based on the method for local rank of matrix, result see Fig. 1 (e) in vision with numerically all both are good than above.Contrast uses also has lower filtration expense compared with Fig. 1 (d) of large matrix order.

Method based on local rank of matrix changes the proper vector in input picture global space into simplification vector in the feature space of described simplification.

The present embodiment is that image considers 4 characteristic types producing 9D space: 2D coordinate, 3D normal, the 3D texture of the 1D degree of depth and basic light.By these feature normalization because the scope of these features is different.In filter window, the maximin of each feature and original value are that linear mapping is to scope [0,1].

The present embodiment is the box filter that pixel filtrator employs a small-sized width of band, and this is one of conventional filtrator.Use best Epanechnikov core herein | t|<1.When the average sample quantity reaching user and provide, use the iteration of a smallest number, additional samples is here by adaptively sampled distribution.

The filter window employing a 11*11 to all tests is for intermediate iteration and use the filter window of a 19*19 for final reconstruct.At [h _min, h _max] scope in h test 5 different.These 5 values are defined as follows:

[0.2＝h _min,0.4,0.6,0.8,1.0＝h _max]。

Build local regression basic statistical model y=f (x)+∈; Described simplification vector is utilized to obtain input picture f (x) after processing based on the SVD removal noise ∈ that blocks of perturbation theory;

Wherein y represents the input picture with noise ∈; Described noise ∈ is made up of deviation and variance:

Z is described simplification vector;

for shared bandwidth h, feature bandwidth b and z mapping relations;

As shown in Fig. 2 (a), input picture is by 32 unified spp (sample rate of each pixel of smpleperpixel) generations, Fig. 2 (b), Fig. 2 (c), the prediction bandwidth shown in Fig. 2 (d) is very intuitively in front 3 dimensions.Result images Fig. 2 (e) and Fig. 2 (f) are the results not using the present invention two step optimization method, and therefore, the present embodiment applies a wide (hb of user-defined smaller strip _j=0.2) and one comparatively large bandwidth (hb _j=1).By using two step optimization methods, result images Fig. 2 (g) save high light part as less in bandwidth time result, in addition, this enforcement also eliminate spike noise on background picture (in green rectangle) as larger in bandwidth time result.

Each pixel z for the input picture after process arranges the variable quantity of light sample size, and we need to play up image again.

The variable quantity of light sample size $\mathrm{\&Delta;}n\left(z\right)=\frac{\mathrm{\&Delta;}rMSE\left(z\right)}{{\mathrm{\&Sigma;}}_t\mathrm{\&Delta;}rMSE\left(z^t\right)};$

Wherein Δ rMSE (z) is the noise varience of red channel of the input picture after process; ε is used to avoid denominator to be zero;

$\mathrm{\&Delta;}MSE\left(z\right)=MSE\left(z\right)\&times;n{\left(z\right)}^{-\frac{4}{k+4}};$

K is the space of each dimension in the feature space simplified.

By the shared bandwidth h calculated _opt, its reconstructed error is predicted to be herein easily find out in our method with h _optdeviation and variance item bias _h(z) and var _h(z).On the other hand, the MSE (z) of our reconstructing method also reduces with identical speed.We arrange sample size Δ n (z) for pixel z subsequently, according to its attenuation rate relative to all pixel quantities.In other words, we are sampled as pixel z by low contradiction and generate Δ n (z) sample, and this is a generally selection used.

Filtrator based on pixel is very common concerning a lot of image filtering method.The present embodiment homing method can use the light sample stored directly to implement, and may provide better result when sacrificing internal memory and computing cost.Contrary, to each characteristic type of each pixel, only store mean value and variance.Use these values being stored in pixel place as our sample under filter window.As a result, need internal memory and performance cost independent of light sample size.When using GPU to go the image (such as, SanMiguel) to a 1k*1k to implement reconstruct, it takes the processing time of about 2s and 7s respectively in interstage and terminal stage.Main Bottleneck is the matrix manipulation of its complexity.

For the benchmark of complexity, such as, SanMiguel scene in Fig. 3 (a), generating 128spp needs about 665s.As long as the expense of result the present embodiment reconstructing method contrast overall render process sub-fraction and produce visually gratifying result efficiency than generation more sample high, this is because it is in the effect of high level error areal distribution sample.The computing cost of distinct methods can change according to benchmark.On average, each sample of method to SURE and NLM of this enforcement has the expense of 14% and 4% respectively.But powerful error prediction and better re-configurability result in the more high-level efficiency when reducing error, and this obtains checking in the contrast of same time.

As shown in Fig. 4 (a), provide a noisy image border, the derivative Fig. 4 (b) and bandwidth Fig. 4 (c) of prediction is for computed image marginal position, when there being very detail edge (object of such as crinosity) to need to be reconstructed, that is instructed by geometry can lose the linear-apporximation of details as Fig. 4 (d), particularly when sample size is relatively low based on pixels approach.As based on sample and the mean method based on pixels approach, method of the present invention can directly be applied on sub-pixel by user, namely a pixel is divided into multiple sub-pixel, and this was proved to be, and finally can obtain result Fig. 4 (e) of the present embodiment method.

Claims (6)

1., based on an Adapti ve rendering method for weighting local regression, it is characterized in that, comprising:

Step 1, carries out the feature space of the simplification of truncated singular value decomposition calculating input image for input picture;

Step 2, the method based on local rank of matrix changes the proper vector in input picture global space into simplification vector in the feature space of described simplification;

Step 3, builds local regression basic statistical model y=f (x)+∈; Described simplification vector is utilized to obtain input picture f (x) after processing based on the SVD removal noise ∈ that blocks of perturbation theory;

Step 4, each pixel z for the input picture after process arranges the variable quantity of light sample size, then plays up.

2., as claimed in claim 1 based on the Adapti ve rendering method of weighting local regression, it is characterized in that, in local regression basic statistical model y=f (x)+∈, wherein y represents the input picture with noise ∈; Described noise ∈ is made up of deviation and variance:

Z is described simplification vector;

for the mapping relations of shared bandwidth h, feature bandwidth b and z.

3., as claimed in claim 2 based on the Adapti ve rendering method of weighting local regression, it is characterized in that, the variable quantity of described light sample size

Wherein:

Δ rMSE (z) is the noise varience of the red channel of the input picture after process;

ε is used to avoid denominator to be zero;

$\mathrm{\&Delta;}MSE\left(z\right)=MSE\left(z\right)\&times;n{\left(z\right)}^{-\frac{4}{k+4}};$

K is the space of each dimension in the feature space simplified.

4., as claimed in claim 3 based on the Adapti ve rendering method of weighting local regression, it is characterized in that,

In formula:

B is diagonal matrix, is expressed as

Trace () is matrix trace;

it is Hesaian matrix.

5., as claimed in claim 4 based on the Adapti ve rendering method of weighting local regression, it is characterized in that,

In formula:

N (z) represents the sample size at z place;

K is the space of each dimension in the feature space simplified;

B _jfor feature bandwidth.

6., as claimed in claim 5 based on the Adapti ve rendering method of weighting local regression, it is characterized in that, b _jfor

F is the mapping relations of b and z;

Z simplifies the vectorial z in local space.