CN102663688B - Surface wave transformation video denoising method based on neighborhood threshold classification - Google Patents

Abstract

The invention discloses a surface wave transformation video denoising method based on neighborhood threshold classification, which is mainly used for improving the phenomenon of image edge blur and less full noise removal. The achieving process is, (1) inputting a noise-containing video coefficient and performing surface wave transformation of same, (2) computing the noise variance of sub band coefficient in each layer each direction after surface wave transformation, (3) computing threshold value of the sub band coefficient in each layer each direction, (4) classifying the sub band coefficient in each layer each direction by utilizing threshold and neighborhood energy, (5) computing signal variance of the sub band coefficient in each layer each direction after classification, (6) performing shrink processing to the coefficient by the signal variance of the sub band coefficient in each layer each direction after classification, (7) performing inverse surface wave transformation of the processed coefficient in order to obtain denoised video image. Compared with prior art, the surface wave transformation video denoising method substantially enhances denoising effect and noise inhibition capability in video image and better maintains detailed information in video image and smooth effect of moving objects.

Description

Surface wave conversion video denoising method based on neighboring thresholding classification

Technical field

The invention belongs to image processing field, relate to image denoising, can be used for video image, the denoising of Biomedical Image and 3-D view.

Background technology

The mankind are the information of obtaining by vision system to the understanding overwhelming majority of objective world.Visual information has played extremely important effect in the process in human perception and the understanding world, but in the vision signal touching at us, often adulterating various noises, to such an extent as to video thickens, Quality Down, thereby cause some the material particular information dropouts in video.In the time that video image is processed or is applied, how to retain the useful information in video image, how the curved surface in captured video image is unusual, is that a focus is also a difficult point.

The research of video denoising method is to process frame by frame taking image as unit at first, and traditional video denoising method is to divide by spatial domain, time domain, transform domain.Airspace filter has the filtering method such as medium filtering and coefficient auto adapted filtering, all can obtain good filter effect to each two field picture.But in Video Applications, because airspace filter does not make full use of time-domain information, can not obtain desirable filter effect.Time-domain filtering has been considered the correlativity between each frame, but is only suitable for static target, can produce the phenomenons such as artifact to moving target.Transform domain is all denoising frame by frame conventionally.Video sequence not only will be paid close attention to the visual effect of each two field picture, also will pay close attention to the visual experience of whole sequence.Therefore, have higher requirement for the denoising of video sequence.

1992, first moving party was to the concept of bank of filters DFB for Bambeger and Smith, and DFB can decompose 2D signal travel direction effectively.2005, Do and Vetterli combined Laplacian pyramid and DFB, the wavelet transformation Contourlet that design makes new advances.But due to the indivisibility of DFB, it be there is no all the time from two-dimensional expansion to multidimensional to perfect implementation method.Until 2005, Yue Lu and M.N.Do propose a kind of new multi-dimensional direction filter set designing method---NDFB (N-dimensional Directional Filter Banks).NDFB adopts a kind of simple, efficient tree structure, can decompose the signal travel direction of any dimension.By adopting one group of iterative filter group, can realize completely and rebuilding, and only have N doubly for the redundance of N dimensional signal.Meanwhile, Yue Lu and M.N.Do, on the basis of NDFB, have proposed surface wave conversion.First surface wave conversion carries out multiple dimensioned decomposition to catch unusual variation to signal, then by NDFB, unidirectional unusual variation is combined into a coefficient.It can catch and represent that the curved surface in high dimensional signal is unusual effectively, is applicable to very much Video processing.For example video can be regarded as two-dimensional space information and the synthetic three-dimensional space-time signal of one dimension temporal information, and the surface of moving objects in video is that curved surface is unusual in this three dimension system.Experimental result shows, some application as Video processing in, the Performance Ratio classic method based on surface wave mapping algorithm has a distinct increment.In some other application, as video compress, 3 D medical image processing and three-dimensional data compression etc., surface wave conversion has good application prospect.

Adopting threshold process is a kind of method of the most frequently used image denoising.Choosing of threshold value is usually the key in this algorithm, and the too small meeting that threshold value is chosen causes removing fully noise; On the contrary, the conference excessively that threshold value is chosen produced and strangled phenomenon signal, and the part important information of lossing signal, can cause the video image after denoising to cross blooming.In transform domain, conventional threshold method has the 3 σ methods that the people such as Q.Pan propose, the BayesShrink method based on minimum risk that M.Vetterli and B.Yu propose.These threshold methods can not obtain desirable denoising effect in details and the abundant video image denoising of background information.Utilizing surface wave transfer pair video image to carry out in the process of denoising, traditional threshold method can not be removed noise fully, the phenomenon such as tend to cause that edge fog and material particular information are strangled.

Summary of the invention

The object of the invention is to for above-mentioned existing methods shortcoming, proposed a kind of surface wave based on neighboring thresholding classification and changed video denoising method, to improve the inadequate phenomenon of soft edge and noise remove, improve the denoising effect of video.

For achieving the above object, the present invention includes following steps:

(1) input, containing noisy video coefficients, is carried out surface wave conversion to video coefficients, and video coefficients is decomposed into 4 layers, and every layer of corresponding direction number is respectively 192,192,48,12;

(2) coefficient after effects on surface wave conversion, calculates the noise variance of detailed level to rough layer all directions subband as follows:

${\mathrm{\σ}}_{l,m}^2={\mathrm{\σ}}_{1,m}^2\·e^{1-l^{1.2}},$

Wherein, be the noise variance of l layer m directional subband, l represents l layer, l ∈ 1,2,3,4}, m represents m directional subband, in layers 1 and 2 subband: m ∈ 1,2 ..., 192}, in the 3rd straton band: m ∈ 1,2 ..., 48}, in the 4th straton band: m ∈ 1,2 ..., 12}, σ _{1, m}poor for the noise criteria of ground floor m directional subband, the poor value of this noise criteria is estimated by the intermediate value estimation technique;

(3) calculate the threshold value of each layer of all directions subband:

$T_{l,m}=r\·\frac{{\mathrm{\σ}}_{l,m}^2}{{\mathrm{\δ}}_{l,m}}$

Wherein, T _{l, m}be the threshold value of l layer m directional subband, the signal standards that is l layer m directional subband is poor, var (w _{l, m}) be l layer m directional subband energy and, r for regulate parameter, get $r=\sqrt{2};$

(4) utilize each layer of all directions sub-band coefficients after the threshold value effects on surface wave conversion obtaining in step (3) to classify:

$y_{l,m}(i,j,k)=\left\{\begin{array}{cc}0,& \mathrm{if}{S}_{l,m}(i,j,k){<T}_{l,m}^2\\ w_{l,m}(i,j,k)& \mathrm{else}\end{array}\right.$

Wherein, y _{l, m}(i, j, k) is classification coefficient after treatment, w _{l, m}(i, j, k) represents coefficient corresponding to position (i, j, k) in l layer m directional subband, $S_{l,m}(i,j,k)=\underset{(o,p,q)\&Element;N_{l,m}(i,j,k)}{\mathrm{\&Sigma;}}{w}_{l,m}^2(o,p,q)$ Neighborhood window energy, N _{l, m}(i, j, k) is with noisy video coefficients w _{l, m}the neighborhood window of centered by (i, j, k) 5 × 5 × 5, (o, p, q) is coordinate corresponding to coefficient in neighborhood window;

(5) signal variance of each layer of all directions sub-band coefficients after calculating classification:

${\mathrm{\&eta;}}_{l,m}^2(i,j,k)=\max ((\frac{1}{\left|N_{l,m}^{\&prime;}(i,j,k)\right|}\&times;\underset{(o_1,p_1,q_1)\&Element;N_{l,m}^{\&prime;}(i,j,k)}{\mathrm{\&Sigma;}}{y}_{l,m}^2(i,j,k))-{\mathrm{\&sigma;}}_{l,m}^2,0)$

Wherein, be the signal variance of processing coefficient of classifying in l layer m directional subband, N ' _{l, m}(i, j, k) is the coefficient y after treatment that classifies _{l, m}the square window of centered by (i, j, k) 5 × 5 × 5, (o ₁, p ₁, q ₁) be coordinate corresponding to coefficient in square window, | N ' _{l, m}(i, j, k) | represent the number of coefficient in square window, max () represents to ask for peaked function;

(6) utilize the signal variance that step (5) obtains to carry out shrink process to each layer of all directions sub-band coefficients after classifying:

${\hat{X}}_{l,m}(i,j,k)=y_{l,m}(i,j,k)\&times;\frac{{\mathrm{\&eta;}}_{l,m}^2(i,j,k)}{{\mathrm{\&eta;}}_{l,m}^2(i,j,k)+{\mathrm{\&sigma;}}_{l,m}^2}$

Wherein, for the video coefficients after shrinking;

(7) coefficient after shrink process is carried out to surface wave inverse transformation, obtain the video image after denoising.

The present invention has the following advantages compared with prior art:

1. the present invention is three-dimensional surface wave conversion due to what adopt, compared with using the two-dimensional transforms such as small echo, Contourlet more direct and convenient aspect video denoising, need not consider the motion compensation relation between each two field picture in video, can effectively catch and represent smooth surface Signal Singularity simultaneously;

2. the present invention simply utilizes threshold value to shrink each layer of all directions sub-band coefficients after converting, but first by neighboring region energy and threshold value, coefficient is classified, then coefficient is carried out to shrink process, therefore there is stronger inhibition ability for the noise of video, also can be good at retaining the detailed information of video image and the smooth effect of moving object simultaneously, the problem that efficiently solves exist in traditional algorithm fuzzy, flicker is significantly improved in visual effect.

Brief description of the drawings

Fig. 1 is realization flow figure of the present invention;

Fig. 2 is the comparison diagram after using the present invention and existing three kinds of methods to Mobile video denoising;

Fig. 3 is the comparison diagram after using the present invention and existing three kinds of methods to Coastguard video denoising.

Embodiment

With reference to Fig. 1, specific embodiment of the invention process is as follows:

Step 1. input, containing noisy video coefficients, is carried out surface wave conversion to video coefficients:

1a) input, containing noisy video coefficients, is divided into its frequency spectrum by the non-sampling filter group of triple channel the subband of three hourglass shape;

1b) hourglass shape subband is decomposed by two-dimensional directional bank of filters travel direction at twice, video coefficients is decomposed into 4 layers, every layer of corresponding direction number is respectively 192,192,48,12.

Coefficient after step 2. effects on surface wave conversion, calculates the noise variance of each layer of all directions subband:

2a) the poor σ of noise criteria of calculating ground floor m directional subband _{1, m}:

σ _1，m＝median(ω _1，m(i，j，k)|)/0.6745，

Wherein, m represents m directional subband, in layers 1 and 2 subband: m ∈ 1,2 ..., 192}, in the 3rd straton band: m ∈ 1,2 ..., 48}, in the 4th straton band: m ∈ 1,2 ..., 12}, ω _{1, m}(i, j, k) is ground floor m directional subband coefficient, and median () is for asking for median;

2b) the noise variance of calculating l layer m directional subband

${\mathrm{\&sigma;}}_{l,m}^2={\mathrm{\&sigma;}}_{1,m}^2\&CenterDot;e^{1-l^{1.2}},$

Wherein, l represents l layer, l ∈ { 1,2,3,4}.

Step 3. is calculated the threshold value of each layer of all directions subband:

3a) energy and the var (ω of calculating l layer m directional subband _{l, m}):

$\mathrm{var}\left({\mathrm{\&omega;}}_{l,m}\right)=\frac{\left(\underset{i=1}{\overset{I}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{J}{\mathrm{\&Sigma;}}}\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{\mathrm{\&omega;}}_{l,m}^2(i,j,k)\right)}{\mathrm{IJK}-1},$

Wherein, ω _{l, m}(i, j, k) is (i, j, k) locational coefficient in l layer m directional subband, i ∈ 1,2 ..., I}, j ∈ 1,2 ..., J}, k ∈ 1,2 ..., and K}, I, J, K are respectively subband length, width and height;

3b) signal standards of calculating l layer m directional subband is poor

3c) the threshold value T of calculating l layer m directional subband _{l, m}:

$T_{l,m}=r\&CenterDot;\frac{{\mathrm{\&sigma;}}_{l,m}^2}{{\mathrm{\&delta;}}_{l,m}},$

Wherein, r, for regulating parameter, gets

Step 4. utilizes the coefficient of each layer of all directions subband after the threshold value effects on surface wave conversion obtaining in step (3) to classify:

4a) the neighborhood window energy S of calculating l layer m directional subband coefficient _{l, m}(i, j, k):

$S_{l,m}(i,j,k)=\underset{(o,p,q)\&Element;N_{l,m}(i,j,k)}{\mathrm{\&Sigma;}}{w}_{l,m}^2(o,p,q),$

Wherein, N _{l, m}(i, j, k) is to contain noisy video coefficients ω _{l, m}the neighborhood window of centered by (i, j, k) 5 × 5 × 5, (o, p, q) is neighborhood window N _{l, m}coordinate corresponding to coefficient in (i, j, k);

4b) utilize threshold value and the neighborhood window energy that step (3) and step (4a) obtain to classify to l layer m directional subband coefficient:

$y_{l,m}(i,j,k)=\left\{\begin{array}{cc}0,& \mathrm{if}{S}_{l,m}(i,j,k)<{T_{l,m}}^2\\ {\mathrm{\&omega;}}_{l,m}(i,j,k)& \mathrm{else}\end{array}\right.$

Wherein, y _{l, m}(i, j, k) is classification coefficient after treatment.

Step 5. is calculated the signal variance of the rear l layer m directional subband coefficient of classification

${\mathrm{\&eta;}}_{l,m}^2(i,j,k)=\max [(\frac{1}{\left|N_{l,m}^{\&prime;}(i,j,k)\right|}\&times;\underset{(o_1,p_1,q_1)\&Element;N_{l,m}^{\&prime;}(i,j,k)}{\mathrm{\&Sigma;}}{y}_{l,m}^2(i,j,k))-{\mathrm{\&sigma;}}_{l,m}^2,0],$

Wherein, N ' _{l, m}(i, j, k) is the coefficient y after treatment that classifies _{l, m}the square window of centered by (i, j, k) 5 × 5 × 5, | N ' _{l, m}(i, j, k) | represent square window N ' _{l, m}the number of coefficient in (i, j, k), (o ₁, p ₁, q ₁) expression square window N ' _{l, m}the position of coefficient in (i, j, k).

Step 6. utilizes the signal variance that step (5) obtains to carry out shrink process to l layer m directional subband coefficient after classifying, and obtains coefficient after treatment

${\hat{X}}_{l,m}(i,j,k)=y_{l,m}(i,j,k)\&times;\frac{{\mathrm{\&eta;}}_{l,m}^2(i,j,k)}{{\mathrm{\&eta;}}_{l,m}^2(i,j,k)+{\mathrm{\&sigma;}}_{l,m}^2},$

Wherein, y _{l, m}(i, j, k) is classification coefficient after treatment, the signal variance of l layer m directional subband coefficient after classification is processed, it is the noise variance of l layer m directional subband.

The coefficient of step 7. after to shrink process carry out surface wave inverse transformation, obtain the video image after denoising.

Effect of the present invention can further illustrate by following experiment:

1. experiment condition:

Experiment simulation environment is: MATLAB R2010a, CPUAMD Athlon2 P320 Dual-Core × 2.10GHz, internal memory 2G, Windows 7.

2. experiment content:

3 σ denoising methods (ST-3 σ), Bayes threshold method (Bayes) and the 3D-CMST method based on hard-threshold function based on surface wave conversion are only processed to the meticulousst two-layer being applied to respectively in surface wave conversion video image is carried out to denoising, compare by the present invention and above-mentioned three kinds of denoising methods.

Experiment 1: in the former Mobile video that is 192 × 192 × 192 in size as shown in Fig. 2 (a), adding standard deviation is 30 Gaussian noise, adds video image after making an uproar as shown in Fig. 2 (b).

Noisy video image shown in Fig. 2 (b) is carried out to denoising by the present invention and above-mentioned three kinds of denoising methods, comparison diagram after denoising is as shown in Fig. 2 (c)-(f), wherein Fig. 2 (c) is the video image after the denoising of ST-3 σ method, Fig. 2 (d) is the video image after the denoising of Bayes method, Fig. 2 (e) is the video image after the denoising of 3D-CMST method, and Fig. 3 (f) is the video image after denoising of the present invention.

Experiment 2: in the former Coastguard video that is 192 × 192 × 192 in size as shown in Fig. 3 (a), adding standard deviation is 30 Gaussian noise, adds video image after making an uproar as shown in Fig. 3 (b).

Noisy video image shown in Fig. 3 (b) is carried out to denoising by the present invention and above-mentioned three kinds of denoising methods, comparison diagram after denoising is as shown in Fig. 3 (c)-(f), wherein Fig. 3 (c) is the video image after the denoising of ST-3 σ method, Fig. 3 (d) is the video image after the denoising of Bayes method, Fig. 3 (e) is the video image after the denoising of 3D-CMST method, and Fig. 3 (f) is the video image after denoising of the present invention.

3. experimental result and analysis

Can find out from Fig. 2 (c) and Fig. 2 (e), for Mobile video, the video image after the denoising obtaining by ST-3 σ method and 3D-CMST method there will be ringing; Can find out from Fig. 2 (d), video image after the denoising obtaining by Bayes method has excessively smoothly caused edge fog phenomenon, can find out from Fig. 2 (f), video image after the denoising that the present invention obtains is relatively level and smooth, noise in video image be can effectively suppress, detailed information and marginal information retained better; From Fig. 3 (c)-(f) can find out, for Coastguard video, all can there is motion blur phenomenon in the denoising video image of the present invention and above-mentioned three kinds of methods, but the video image after denoising of the present invention is subject to motion blur phenomenon to affect minimum, the effect that the present invention simultaneously obtains on detailed information and Edge preserving is best, inhibition ability to noise is also the strongest, and noise remove insufficient in other three kinds of methods there will be scintillation.Because Figure of description of the present invention only can provide static image, it is more outstanding that in actual video display process, the present invention can eliminate the effect of smear and scintillation.

The present invention adopts PSNR value as objective evaluation index, and this PSNR value is the mean value of each two field picture PSNR after video image denoising, the quality of major embodiment image; To Coastguard video image in the Mobile video image in Fig. 2 (a) and Fig. 3 (a), adding respectively standard deviation is 20,30,40,50 Gaussian noise, the PSNR value of the denoising rear video image obtaining by the present invention and above-mentioned three kinds of methods, as shown in table 1.

The video image PSNR value that table 1. obtains after to video image denoising by the present invention and above-mentioned three kinds of methods

As can be seen from the above table, compared with existing three kinds of denoising methods, all there is raising by a relatively large margin by the PSNR value of the video image after denoising of the present invention.

To sum up, the conversion of the surface wave based on the neighboring thresholding classification video denoising method that the present invention proposes, in effectively removing noise, can keep well the edge details information of video image, therefore method proposed by the invention has stronger inhibition ability for the noise of video, also can be good at retaining the detailed information of video image and the smooth effect of moving object, the problem that has effectively solved exist in traditional algorithm fuzzy, flicker is significantly improved the video image after denoising in visual effect.

Claims (2)

1. the conversion of the surface wave based on a neighboring thresholding classification video denoising method, comprises following steps:

(1) input, containing noisy video, is carried out surface wave conversion to noisy video, obtains video coefficients, and video coefficients is decomposed into 4 layers, and every layer of corresponding direction number is respectively 192,192,48,12;

(2) coefficient after effects on surface wave conversion, calculates the noise variance of detailed level to rough layer all directions subband as follows:

${\mathrm{\&sigma;}}_{l,m}^2={\mathrm{\&sigma;}}_{1,m}^2\&CenterDot;e^{1-l^{1.2}},$

Wherein, be the noise variance of l layer m directional subband, l represents l layer, l ∈ 1,2,3,4}, m represents m directional subband, in layers 1 and 2 subband: m ∈ 1,2 ..., 192}, in the 3rd straton band: m ∈ T, 2 ..., 48}, in the 4th straton band: m ∈ 1,2 ..., 12}, σ _{1, m}poor for the noise criteria of ground floor m directional subband, the poor value of this noise criteria is estimated by the intermediate value estimation technique;

(3) calculate the threshold value of each layer of all directions subband:

$T_{l,m}=r\&CenterDot;\frac{{\mathrm{\&sigma;}}_{l,m}^2}{{\mathrm{\&delta;}}_{l,m}},$

Wherein, T _l,mbe the threshold value of l layer m directional subband, the signal standards that is l layer m directional subband is poor, var (w _l,m) be l layer m directional subband energy and, r for regulate parameter, get

(4) utilize each layer of all directions sub-band coefficients after the threshold value effects on surface wave conversion obtaining in step (3) to classify:

$y_{l,m}(i,j,k)=\left\{\begin{array}{c}0,\mathrm{if}{S}_{l,m}(i,j,k)<T_{l,m}^2\\ w_{l,m}(i,j,k),\mathrm{else}\end{array}\right.$

Wherein, y _l,m(i, j, k) is classification coefficient after treatment, w _l,m(i, j, k) represents coefficient corresponding to (i, j, k) in l layer m directional subband, $S_{l,m}(i,j,k)=\underset{(q,p,q)\&Element;N_{l,m}(i,j,k)}{\mathrm{\&Sigma;}}{w}_{l,m}^2(o,p,q)$ Neighborhood window energy, N _l,m(i, j, k) is with noisy video coefficients w _l,mthe neighborhood window of centered by (i, j, k) 5 × 5 × 5, (o, p, q) is coordinate corresponding to coefficient in neighborhood window;

(5) signal variance of each layer of all directions sub-band coefficients after calculating classification:

${\mathrm{\&eta;}}_{l,m}^2(i,j,k)=\max ((\frac{1}{|N_{l,m}^{\&prime;}(i,j,k)}\&times;\underset{(q_1,p_1,q_1)\&Element;N_{l,m}^{\&prime;}(i,j,k)}{\mathrm{\&Sigma;}}{y}_{l,m}^2(i,j,k))-{\mathrm{\&sigma;}}_{l,m}^2.0)$

Wherein, be the signal variance of processing coefficient of classifying in l layer m directional subband, N ' _l,m(i, j, k) is the coefficient y after treatment that classifies _l,mthe square window of centered by (i, j, k) 5 × 5 × 5, (o ₁, p ₁, q ₁) be coordinate corresponding to coefficient in square window, | N ' _l,m(i, j, k) | represent the number of coefficient in square window;

(6) utilize the signal variance that step (5) obtains to carry out shrink process to each layer of all directions sub-band coefficients after classifying:

${\hat{X}}_{l,m}(i,j,k)=y_{l,m}(i,j,k)\&times;\frac{{\mathrm{\&eta;}}_{l,m}^2(i,j,k)}{{\mathrm{\&eta;}}_{l,m}^2(i,j,k)+{\mathrm{\&sigma;}}_{l,m}^2}$

Wherein, for the video coefficients after shrinking;

(7) coefficient after shrink process is carried out to surface wave inverse transformation, obtain the video image after denoising.

2. method according to claim 1, wherein step (1), carry out as follows:

First, input, containing noisy video coefficients, is divided into its frequency spectrum by the non-sampling filter group of triple channel the subband of three hourglass shape;

Secondly, hourglass shape subband is decomposed by two-dimensional directional bank of filters travel direction at twice, video coefficients is decomposed into 4 layers, every layer of corresponding direction number is respectively 192,192,48,12.