CN102238316A - Self-adaptive real-time denoising scheme for 3D digital video image - Google Patents

Abstract

The invention provides a self-adaptive real-time denoising scheme for a 3D digital video image, which is used for filtering and denoising a video image in a space domain two-dimensional mode and a time domain one-dimensional mode. The method comprises the following steps of: detecting the motion intensity of the video image, estimating a noise standard difference of the image and setting two different threshold value limits according to an estimation result of the noise standard difference; and adjusting parameters in time domain filtering in a self-adaptive mode according to a relation between the motion intensity and the limits and weighting a time domain filtering result and a space domain filtering result according to the relation in a self-adaptive mode to obtain a denoised output result. In the scheme, the computational load and memory space are small, the 3D digital video image can be denoised effectively in real time, borders and details of the image are protected, and the watching comfort of the 3D digital video image is enhanced greatly.

Description

The real-time noise reduction scheme of a kind of self adaptation of 3D digital video image

Technical field

The present invention relates to the digital video image process field, specifically, the real-time noise reduction scheme of self adaptation that relates to a kind of 3D digital video image is mainly used in the occasion that needs noise reduction in the processing of 3D digital video image, transmission, the demonstration, particularly needs the occasion of real-time noise reduction.

Background technology

In recent years, the 3D digital video image has obtained using for example multimedia service, visual telephone, video broadcasting, video tracking etc. widely in growing field.The obtaining of 3D digital video image, compress, in the link such as transmission, reception, all introduce noise inevitably.If these noises are filtering effectively in addition not, not only can have influence on the visual effect of video image, and bring adverse influence can for subsequent treatment link (for example compressed encoding etc.).Therefore, it is very necessary studying the noise scheme that effectively reduces in the 3D digital video image.

Generally, the noise in the 3D digital video image is considered to and image independent Gaussian white noise, and the noise that is characterized in each two field picture is separate, and the noise of each two field picture occurs at random, and the obedience average is zero Gaussian Profile.On the contrary, the content of each frame is a height correlation in the 3D video image, is consistent along the movement locus of object direction in time domain, does not change between every frame.Therefore, can utilize the correlation of picture material and the irrelevance design noise reduction scheme of noise, from the image that contains noise, recover to obtain original image as far as possible.

At present, the noise reduction scheme of 3D digital video image can be divided into following a few class: the first kind is the spatial domain noise reduction, and it only utilizes each two field picture that video image is done noise reduction process; Second class is the time domain noise reduction, and it only utilizes the motion compensation of image that video image is done noise reduction process; The 3rd class is a Space Time territory noise reduction, and promptly usually said 3D noise reduction is done noise reduction process in spatial domain and time domain to video image simultaneously.The spatial domain noise reduction is relative simple with the operation of time domain noise reduction, but occurs problems such as edge quality is poor, blocking effect easily.The 3D noise reduction combines the advantage of the two, has made full use of the information of 3D digital video image, thereby has obtained increasing concern.

Existing patent has proposed some 3D noise reduction schemes, as the recursive filtering scheme of the compensation filter scheme of FIR filtered version, linear Kalman filtering (perhaps EKF) form.These schemes mainly exist amount of calculation and the excessive problem of memory space.In order to obtain filtering parameter preferably, often need a lot of reference frame (the preceding p frame and the back p frame that need present frame).This needs bigger storage resources expense on the one hand, makes the noise reduction computing slow excessively on the other hand, thereby is not suitable for the real time video image noise reduction.In addition, these filtering algorithms can cause the accumulation of error and the amplification in the filtering, thereby finally influence the performance of filtering algorithm.

According to above analysis as can be seen, how to utilize the information of 3D digital video image on the time domain and spatial domain, dexterously time domain noise reduction and spatial domain noise reduction are combined, the problem of avoiding single noise reduction mode to produce is improved the amount of calculation of existing 3D noise reduction simultaneously and memory space is excessive and error accumulation problem is the key of design 3D digital video image noise reduction scheme.

Summary of the invention

The invention provides a kind of real-time noise reduction scheme of self adaptation of 3D digital video image, be mainly used in the occasion that needs real-time noise reduction in the processing of 3D digital video image, transmission, the demonstration, may further comprise the steps:

Step 1: present frame is moved into previous frame, and from the video image input, read the current frame image for the treatment of noise reduction again;

Step 2: current frame image is divided into the plurality of sub piece, utilizes this a little noise criteria difference to estimate to present frame;

Step 3: utilize sub-piece that step 2 obtains and the sub-piece in the previous frame to carry out match search, obtain match block;

Step 4: two different threshold values are set according to the noise criteria difference that obtains in the step 2;

Step 5: image motion intensity is estimated according to the match block that obtains in the step 3;

Step 6: two threshold values that obtain in the image motion intensity that obtains in the step 5 and the step 4 are compared, select the coefficient of time-domain filtering by the two magnitude relationship;

Step 7: two threshold values that obtain in the image motion intensity that obtains in the step 5 and the step 4 are compared, select one group of weight coefficient by the two magnitude relationship;

Step 8: utilize the time-domain filtering coefficient of selecting in the step 6 that current frame image is carried out time-domain filtering;

Step 9: utilize the noise criteria that obtains in the step 2 poor, current frame image is carried out airspace filter;

Step 10: the weight coefficient of selecting in 7 is weighted the result who obtains in step 8 and the step 9 set by step;

Step 11: the noise reduction result of output current frame image.

The effect explanation

The present invention proposes a kind of real-time noise reduction scheme of self adaptation of 3D digital video image, it carries out the filtering noise reduction process to image on the two peacekeeping time domain one dimensions of spatial domain.This method is carried out exercise intensity to image and is detected, and the noise criteria difference of image is estimated, a kind of method of estimated result and the image motion intensity detection setting threshold thresholding according to the noise criteria difference is provided.

The invention provides a kind of by adjusting the method for the parameter in the time-domain filtering with the self adaptation that concerns of exercise intensity and thresholding; provide a kind of simultaneously, made that edge of image and details have obtained protection by the result of time-domain filtering and airspace filter being carried out adaptive weighted method as the output result behind the noise reduction with the self adaptation that concerns of exercise intensity and thresholding.

The present invention can carry out noise reduction to the 3D digital video image effectively, has greatly increased the comfortableness that the 3D digital video image is watched.Simultaneously, the present invention only needs the image of present frame and previous frame in the time domain noise reduction, greatly reduce amount of calculation and memory space, thereby is particularly suitable for carrying out the 3D digital video image occasion of real-time noise reduction.

Accompanying drawing and subordinate list explanation

Fig. 1: the real-time noise reduction scheme of the self adaptation of 3D digital video image provided by the invention flow chart;

Fig. 2: the time-domain filtering schematic diagram among the present invention;

Fig. 3: the airspace filter schematic diagram among the present invention;

Fig. 4: time-domain filtering among the present invention and airspace filter weighting schematic diagram;

Table 1: the time-domain filtering coefficient among the present invention and the relation of image motion intensity and threshold value;

Table 2: the relation of the weight coefficient among the present invention and image motion intensity and threshold value.

Embodiment

For making the purpose, technical solutions and advantages of the present invention clearer, below in conjunction with specific embodiment, and with reference to accompanying drawing, the present invention is described in more detail.

Fig. 1 is the real-time noise reduction scheme of the self adaptation of a 3D digital video image provided by the invention flow chart.From this figure clearly as can be seen, noise reduction scheme provided by the invention comprises following 11 steps, is followed successively by step 1-step 11.By each step present embodiment is described further below.

Step 1: present frame is moved into previous frame, and from the video image input, read the current frame image for the treatment of noise reduction again

Step 2: current frame image is divided into the plurality of sub piece, utilizes this a little noise criteria difference to present frame to estimate to suppose that pending image size is M * N, being divided into some sizes is the sub-piece of L * L, and wherein L is the common factor of M and N.The sub-piece of each L * L is a basic handling unit.For example, pending image size is 1024 * 768, can get L=32.Then, the noise criteria of calculating each sub-piece respectively is poor, and computing formula is

$\mathrm{\σ}=\sqrt{\frac{1}{L\×L}\underset{i=0}{\overset{L-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{L-1}{\mathrm{\Σ}}}{(f_{\mathrm{ij}}-{\stackrel{\‾}{f}}_{\mathrm{ij}})}^2}---\left(1\right)$

Wherein,

Mean value for this sub-each pixel of piece:

After the noise criteria difference of all sub-pieces is calculated, select of the estimation of their minimum value as the noise criteria difference of current frame image

Select minimum value as estimating it is because the variance in the zone of image relatively flat mainly is the noise variance decision.

Step 3: utilize sub-piece that step 2 obtains and the sub-piece in the previous frame to carry out match search, obtain match block

To the sub-piece of each L * L of present frame, in previous frame, adopt minimum SAD criterion, find the piece of the sad value minimum of the sub-piece of L * L therewith, be called match block, the computational methods of sad value are:

$\mathrm{SAD}=\underset{i=0}{\overset{L-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{L-1}{\mathrm{\Σ}}}|f_{\mathrm{ij}}-r_{\mathrm{ij}}|---\left(2\right)$

Wherein, f _IjBe the pixel value of current sub-block relevant position, r _IjPixel value for last one sub-piece relevant position.Consider memory space and complexity and with the weighting of airspace filter, traditional " three-step approach " selected in the piece matching operation, promptly selects 4,2,1 step-size in search for use, knows personnel in its detailed process this area and all should be appreciated that, do not repeat them here.

Step 4: two different threshold values are set according to the noise criteria difference that obtains in the step 2

Poor according to noise criteria

Estimated value setting threshold TH ₁And TH ₂, adjust parameter and the weight coefficient in the time-domain filtering, TH in the present embodiment according to the self adaptation that concerns of these two threshold values and image motion intensity ₁And TH ₂Poor with the noise criteria of estimating Relation in direct ratio, promptly

${\mathrm{<figure-callout\; id="1"\; label="TH"\; filenames="HSA00000085880000021.png"\; state="\{\{state\}\}">TH</figure-callout>}}_1={\mathrm{<figure-callout\; id="1"\; label="A"\; filenames="HSA00000085880000021.png"\; state="\{\{state\}\}">A</figure-callout>}}_1\widehat{\mathrm{\&sigma;}},$ ${\mathrm{<figure-callout\; id="2"\; label="TH"\; filenames="HSA00000085880000031.png"\; state="\{\{state\}\}">TH</figure-callout>}}_2={\mathrm{<figure-callout\; id="2"\; label="A"\; filenames="HSA00000085880000031.png"\; state="\{\{state\}\}">A</figure-callout>}}_2\widehat{\mathrm{\&sigma;}}---\left(3\right)$

A ₁And A ₂Be two positive numbers, by a large amount of emulation testings, selected A ₁=1.8, A ₂=1.2.

Step 5: image motion intensity is estimated according to the match block that obtains in the step 3

Each sub-piece exercise intensity estimated value K _pTry to achieve according to following formula:

$K_p=\frac{\mathrm{SAD}}{L\&times;L}---\left(4\right)$

Wherein, sad value calculates according to (2) formula.

Step 6: two threshold values that obtain in the image motion intensity that obtains in the step 5 and the step 4 are compared, select the coefficient of time-domain filtering by the two magnitude relationship

This coefficient w is according to K _pAnd TH ₁And TH ₂Relation determine that the partial image motion is (K slowly _p＜TH ₂), the general (TH of image motion ₂≤ K _p＜TH ₁) and the violent (K of image motion _p〉=TH ₁) three kinds of situations, concrete numerical value is chosen as follows by emulation testing:

$w=\left\{\begin{array}{cc}0.4& K_p<{\mathrm{<figure-callout\; id="2"\; label="TH"\; filenames="HSA00000085880000031.png"\; state="\{\{state\}\}">TH</figure-callout>}}_2\\ 0.65& {\mathrm{<figure-callout\; id="2"\; label="TH"\; filenames="HSA00000085880000031.png"\; state="\{\{state\}\}">TH</figure-callout>}}_2\&le;K_p<{\mathrm{<figure-callout\; id="1"\; label="TH"\; filenames="HSA00000085880000021.png"\; state="\{\{state\}\}">TH</figure-callout>}}_1\\ 0.9& K_p\&GreaterEqual;{\mathrm{TH}}_1\end{array}\right.---\left(5\right)$

By formula (5) as can be seen, when image motion is violent, select the numerical value of w bigger; On the contrary, when image motion is slow, select the numerical value of w less.In actual Filtering Processing process, deposit above-mentioned numerical value in the table (table 1), realize choosing process by tabling look-up.

Step 7: two threshold values that obtain in the image motion intensity that obtains in the step 5 and the step 4 are compared, select one group of weight coefficient (β by the two magnitude relationship ₁, β ₂)

Weight coefficient is according to K _pAnd TH ₁And TH ₂Relation determine that the partial image motion is (K slowly _p＜TH ₂), the general (TH of image motion ₂≤ K _p＜TH ₁) and the violent (K of image motion _p〉=TH ₁) three kinds of situations.When image motion is violent, airspace filter is added bigger weight, when image motion is slow, time-domain filtering is added bigger weight, concrete numerical value is chosen as follows by emulation testing:

$({\mathrm{\&beta;}}_1,{\mathrm{\&beta;}}_2)=\left\{\begin{array}{cc}\left(\mathrm{0.8,0.2}\right)& K_p<{\mathrm{TH}}_2\\ \left(\mathrm{0.5,0.5}\right)& {\mathrm{<figure-callout\; id="2"\; label="TH"\; filenames="HSA00000085880000031.png"\; state="\{\{state\}\}">TH</figure-callout>}}_2\&le;K_p<\\ \left(\mathrm{0.2,0.8}\right)& K_p\&GreaterEqual;{\mathrm{<figure-callout\; id="1"\; label="TH"\; filenames="HSA00000085880000021.png"\; state="\{\{state\}\}">TH</figure-callout>}}_1\end{array}{\mathrm{TH}}_1\right.---\left(6\right)$

In actual Filtering Processing process, deposit above-mentioned numerical value in the table (table 2), realize choosing process by tabling look-up.

Step 8: utilize the time-domain filtering coefficient of selecting in the step 6 that current frame image is carried out time-domain filtering

Time-domain filtering is that current frame image and previous frame image are carried out the single order recurrence is level and smooth, as shown in Figure 2.Filtering strength is adjusted control according to coefficient w:

${\hat{f}}_1={\mathrm{wf}}_{\mathrm{ij}}+(1-w)r_{\mathrm{ij}}---\left(7\right)$

Wherein,

Be the output result of time-domain filtering, f _IjBe the pixel value of each point in each sub-piece of present frame, r _IjBe the pixel value of each point in each sub-piece of previous frame, w is by K _pAnd TH ₁And TH ₂The value from table 1, found of relation.Know personnel in its detailed process this area and all should be appreciated that, do not repeat them here.

Step 9: utilize the noise criteria that obtains in the step 2 poor, current frame image is carried out airspace filter

Airspace filter can be regarded the spectral window of the long W of being of a window as in the process that contains glide filter on the image of noise, as shown in Figure 3.Know personnel in its detailed process this area and all should be appreciated that, do not repeat them here.Choose W=4 in the present embodiment.The weight coefficient of airspace filter filter is chosen by following (8) formula, and it has taken all factors into consideration the influence of pixel value and distance

$w_{\mathrm{ij}}=\exp [-\frac{{(i-k)}^2+{(j-1)}^2}{2{\mathrm{\&sigma;}}_d^2}]\&CenterDot;\exp [-\frac{{(f_{\mathrm{ij}}-f_{k1})}^2}{2{\mathrm{\&sigma;}}_r^2}]---\left(8\right)$

As can be seen from the above equation, w _IjMultiply each other by two and to obtain.First calculating be the range difference of certain pixel in current pixel and its W * W window, σ _dStandard deviation for range direction; Second calculating be the poor of certain pixel value in current pixel value and its W * W window, σ _rStandard deviation for pixel value.Choose in the present embodiment

The estimated result of airspace filter

Obtain by following formula (9):

${\hat{f}}_2=w_{\mathrm{ij}}\&CircleTimes;f_{\mathrm{ij}}---\left(9\right)$

Wherein, symbol

The expression convolution algorithm.

Step 10: the weight coefficient of selecting in 7 is weighted the result who obtains in step 8 and the step 9 set by step

In order to take into account the result of time-domain filtering and airspace filter, the two is weighted, as shown in Figure 4.Present embodiment is chosen linear method of weighting, has the advantage that is easy to realize, can solve simultaneously problems such as image border behind the noise reduction is of poor quality, blocking effect.Weight (β ₁, β ₂) according to K _pAnd TH ₁And TH ₂Relation from table 2, find, specifically be calculated as follows:

$\hat{f}={\mathrm{\&beta;}}_1{\hat{f}}_1+{\mathrm{\&beta;}}_2+{\hat{f}}_2---\left(10\right)$

Wherein,

As the output result in the following step 11,

With

Be respectively time domain and airspace filter result.

Step 11: the noise reduction result of output current frame image.

Although above the specific embodiment of the present invention is described, clearly, the invention is not restricted to the scope of embodiment.For the one skilled in the art, under the situation of invention scope that non-migration claims are limited and spirit, can also make various modifications and changes to these embodiment.Therefore, specification and accompanying drawing are descriptive, rather than determinate, and all utilize the innovation and creation of thinking of the present invention all should be at the row of protection.

Claims (10)

1. the real-time noise reduction scheme of the self adaptation of a 3D digital video image is characterized in that, may further comprise the steps:

Step 1: present frame is moved into previous frame, and from the video image input, read the current frame image for the treatment of noise reduction again;

Step 2: current frame image is divided into the plurality of sub piece, utilizes this a little noise criteria difference to estimate to present frame;

Step 3: utilize sub-piece that step 2 obtains and the sub-piece in the previous frame to carry out match search, obtain match block;

Step 4: two different threshold values are set according to the noise criteria difference that obtains in the step 2;

Step 5: image motion intensity is estimated according to the match block that obtains in the step 3;

Step 6: two threshold values that obtain in the image motion intensity that obtains in the step 5 and the step 4 are compared, select the coefficient of time-domain filtering by the two magnitude relationship;

Step 7: two threshold values that obtain in the image motion intensity that obtains in the step 5 and the step 4 are compared, select one group of weight coefficient by the two magnitude relationship;

Step 8: utilize the time-domain filtering coefficient of selecting in the step 6 that current frame image is carried out time-domain filtering;

Step 9: utilize the noise criteria that obtains in the step 2 poor, current frame image is carried out airspace filter;

Step 10: the weight coefficient of selecting in 7 is weighted the result who obtains in step 8 and the step 9 set by step;

Step 11: the noise reduction result of output current frame image.

2. as right 1 described method, it is characterized in that, suppose that image size pending in the step 2 is M * N, be divided into several sizes and be the sub-piece of L * L, wherein L is the common factor of M and N.The sub-piece of each L * L is a basic handling unit.Then, the noise criteria of calculating each sub-piece respectively is poor, and selects the estimation of the minimum value of each sub-block noise standard deviation as the noise criteria difference of current frame image

3. as right 1 described method, it is characterized in that, in the step 3, sub-piece to each L * L of present frame, the employing minimum absolute difference is with (Sum of Absolute Differences, SAD) criterion find the piece of the sad value minimum of the sub-piece of L * L therewith in previous frame, be called match block, the computational methods of sad value are:

$\mathrm{SAD}=\underset{i=0}{\overset{L-1}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{L-1}{\mathrm{\&Sigma;}}}|f_{\text{ij}}-r_{\mathrm{ij}}|$

Wherein, f _IjBe the pixel value of current sub-block relevant position, r _IjPixel value for last one sub-piece relevant position.

4. as right 1 described method, it is characterized in that, poor in the step 4 according to noise criteria

Estimated value set two threshold value TH ₁And TH ₂, these two threshold values are all poor with the noise criteria of estimating

Relation in direct ratio, promptly

$T{H}_1=A_1\widehat{\mathrm{\&sigma;}},$ ${\mathrm{TH}}_2=A_2\widehat{\mathrm{\&sigma;}}$

A ₁And A ₂Be two positive numbers (supposition A ₁＞A ₂, so TH ₁＞TH ₂).

5. as right 1 described method, it is characterized in that, in the step 5 each L * L the estimated value K of sub-piece exercise intensity _pTry to achieve according to the sad value between this sub-piece and the match block:

$K_p=\frac{\mathrm{SAD}}{L\&times;L}$

6. as right 1 described method, it is characterized in that the time-domain filtering coefficient w in the step 6 is according to K _pAnd TH ₁And TH ₂Relation determine:

$w=\left\{\begin{array}{cc}0.4& K_p<{\mathrm{TH}}_2\\ 0.65& {\mathrm{TH}}_2\&le;K_p<{\mathrm{TH}}_1\\ 0.9& K_p\&GreaterEqual;{\mathrm{TH}}_1\end{array}\right.$

7. as right 1 described method, it is characterized in that the one group of weight coefficient (β that chooses in the step 7 ₁, β ₂), β wherein ₁Be time-domain filtering weighted factor, β ₂Be the airspace filter weighted factor.β ₁And β ₂Satisfy β ₁+ β ₂=1, concrete numerical basis K _pAnd TH ₁And TH ₂Relation determine:

$({\mathrm{\&beta;}}_1,{\mathrm{\&beta;}}_2)=\left\{\begin{array}{cc}\left(\mathrm{0.8,0.2}\right)& K_p<{\mathrm{TH}}_2\\ (0.5,0.5)& {\mathrm{TH}}_2\&le;K_p<{\mathrm{TH}}_1\\ \left(\mathrm{0.2,0.8}\right)& K_p\&GreaterEqual;{\mathrm{TH}}_1\end{array}\right.$

8. as right 1 described method, it is characterized in that the time-domain filtering intensity in the step 8 is adjusted control according to coefficient w:

${\hat{f}}_1={\mathrm{wf}}_{\mathrm{ij}}+(1-w)r_{\mathrm{ij}}$

Wherein,

Be the output result of time-domain filtering, f _IjBe the pixel value of each point in each sub-piece of present frame, r _IjPixel value for each point in each sub-piece of previous frame.

9. as right 1 described method, it is characterized in that the airspace filter in the step 9 has been taken all factors into consideration the influence of pixel value and distance.Suppose the long W of being of window of airspace filter, then the filter weight coefficient is

$w_{\mathrm{ij}}=\exp [-\frac{{(i-k)}^2+{(j-l)}^2}{2{\mathrm{\&sigma;}}_d^2}]\&CenterDot;\exp [-\frac{{(f_{\mathrm{ij}}-f_{\mathrm{kl}})}^2}{2{\mathrm{\&sigma;}}_r^2}]$

w _IjMultiply each other by two and to obtain.First calculating be the range difference of certain pixel in current pixel and its W * W window, σ _dFor the standard deviation of range direction, choose

Second calculating be the poor of certain pixel value in current pixel value and its W * W window, σ _rFor the standard deviation of pixel value, choose

The estimated result of airspace filter obtains by following formula:

${\hat{f}}_2=w_{\mathrm{ij}}\&CircleTimes;f_{\mathrm{ij}}$

Wherein, symbol

The expression convolution algorithm.

10. as right 1 described method, it is characterized in that, in the step 10 respectively with time domain and airspace filter weighting β as a result ₁And β ₂:

$\hat{f}={\mathrm{\&beta;}}_1{\hat{f}}_1+{\mathrm{\&beta;}}_2{\hat{f}}_2$

Wherein,

For the result is exported in filtering at last,

With

Be respectively time domain and airspace filter result.

Images