CN103501401A - Real-time video de-noising method for super-loud noises based on pre-filtering - Google Patents

Abstract

The invention belongs to the field of video information processing and introduces a pre-filtering idea to provide a real-time video de-noising method for super-loud noises based on pre-filtering. The real-time video de-noising method comprises the following steps: respectively adopting mean value down-sampling and Gaussian filtering to pre-filter an original image; calculating a weight on the basis and using a former frame when weighting and averaging; using a de-noised image to replace the original image to obtain a time domain filtering result; and finally, combining a space filtering result to carry out weighting and averaging again so as to obtain a final de-noised image. According to the real-time video de-noising method provided by the invention, a de-noising effect of an algorithm is improved by adopting the pre-filtering idea; the real-time video de-noising method particularly has a very obvious de-noising effect for the super-loud noises, has good instantaneity and less resource consumption and is easy to realize in hardware equipment.

Description

Real-time video denoising method towards the super large noise based on pre-filtering

Technical field

The invention belongs to the video information process field, relate to video denoising method, relate in particular to a kind of real-time video denoising method of processing based on the image pre-filtering towards the super large noise.

Background technology

One of major issue of digital imagery field face is the noise remove problem.Particularly, under low light conditions, need to increase sensor photosensitive degree ISO.This will bring than the large several times of the normal photographing super large noise of decades of times even.How under low light conditions, still to obtain the good imaging effect of image quality and be one of the developing direction in digital imaging apparatus future.

From the angle of hardware, can adopt the secondary light source Enhanced Imaging quality such as photoflash lamp, infrared lamp, LED light compensating lamp, but their effect is limited.For example, photoflash lamp can't use in continuous video, and infrared lamp only has tens meters in outdoor scene operating distance, and the brightness of LED lamp light filling is inadequate, etc.And, in the application such as safety monitoring, sometimes do not wish to use these secondary light sources and cause own target exposure.On the other hand, be exactly to solve the noise reduction problem from the angle of software algorithm.

Software denoising method for video has directly adopted the two-dimentional denoising method for image at first.Obviously image denoising does not take full advantage of continuous videos relevance in time, and noise reduction is unsatisfactory.At present, people have proposed many denoising methods for the video three-dimensional character.As the VBM3D with transform domain based on time domain piece coupling, the methods such as SURE-LET, the methods such as ST-GSM of the space-time Gauss yardstick mixed model based on wavelet field, the methods such as WAVTHRF based on wavelet transformation and multiscale analysis.From effect, say, these methods all have certain denoising ability, particularly VBM3D and two kinds of methods of SURE-LET.But the ubiquitous problem of these methods is that amount of calculation is large, real-time is poor, and resource consumption is many, is difficult to be applied in digital imaging apparatus.Although people have also done various trials by these algorithm real time implementations, effect is unsatisfactory.As, the people such as Katona have realized a kind of video denoising method based on wavelet field.Although the method denoising effect is general, they have used two FPGA just to realize such algorithm.This has also needed the whole ISP(ISP such as automatic exposure, Automatic white balance, bad point correction, color correction, gamma correction, edge enhancing, high light inhibition, BLC for one, the processing of Image Signal Processing picture signal) digital imaging apparatus, the resource consumed is too many, is difficult to accept.

In fact, for the simplest method of video denoising, be exactly the frame accumulation.If scene is static, and there is no moving object, the frame accumulation will obtain best denoising effect undoubtedly so.But if moving target is arranged, will produce the diplopia phenomenon so.So for video denoising, a very important problem is exactly compensating for variations, comprises motion compensation, illumination variation compensation etc.Considered frame accumulation and compensating for variations, then the real-time of combination algorithm and little computational requirements, a kind of simple and effective video denoising method is according to pixels to be worth Similarity-Weighted.Be that in video, front and back frame same spatial location pixel difference is less, each frame weight is average; Pixel difference is larger, and the present frame weight is larger.If the present frame appearance that changes like this, front and back frame pixel differs greatly, and average weighted result is almost only got the pixel value of present frame, has carried out compensating for variations.In the zone changed, because the past frame that there is no time domain can be used for the frame accumulation, so be aided with again space filtering, denoising is carried out in this zone.The ASTA method that the people such as Bennett propose just is based on this framework.Although the existing method resource consumption based on the pixel value Similarity-Weighted is few, the energy real-time implementation, and has certain denoising effect,, for the denoising of super large noise the time, effect is unsatisfactory.Because the noise under low light conditions, the decades of times of normal noise normally, and the variation under low light conditions, much smaller than the normal illumination condition.It is generally acknowledged that human eye can differentiate the variation of luminance difference at 10 left and right (brightness range 0-255), but when the standard deviation (degree of flashing) of noise 10 when above, existing method will be difficult to distinguish noise and normal variation.In order not cause diplopia, the noise of only withing a hook at the end, thus cause denoising effect not good, limited the application based on pixel value Similarity-Weighted method.

Summary of the invention

The object of the invention is to overcome the existing shortcoming based on pixel value Similarity-Weighted method, propose a kind of effective solution diplopia, denoising effect is good, and real-time is good, consumed resource is little, can be used for the video denoising method of digital imaging apparatus.By the analysis to existing algorithm and other reference methods, find that how correctly trying to achieve weight is the key of dealing with problems.Based on this, the present invention proposes a kind of real-time video denoising method based on pre-filtering.The method has kept the characteristics that real-time is good, resource consumption is few of former method, while having overcome on the other hand in the face of the super large noise, and the problem that denoising effect is not good.

The present invention adopts following technical scheme to realize:

A kind of towards the super large noise real-time video denoising method based on pre-filtering, comprise the following steps:

Step 1: input current frame image Y _i, i means the sequence number of present frame, i=1,2,3 ..., current frame image is carried out to gaussian filtering, obtain the space gaussian filtering result images of original resolution

; Current frame image is done to average down-sampled simultaneously, obtain the low-resolution image of average after down-sampled; Described average is down-sampled is that current frame image is divided into to down-sampled of several formed objects, usings the mean value of all pixels in described down-sampled as this sampled value of down-sampled;

Step 2: the low-resolution image after down-sampled to described average carries out gaussian filtering, obtains low-resolution spatial gaussian filtering result images P _i, i means the sequence number of present frame, i=1,2,3

Step 3: the low-resolution spatial gaussian filtering result images P of storage after step 2 is processed _i; Judge that whether present frame surpasses the N frame, when i>N, will be on the time when storage apart from present frame, a frame farthest substitutes, only storage comprises present frame P _iand the N frame adjacent image { P before present frame _i-N, P _i-N+1..., P _i-1; Described N is predefined value;

Step 4: judge whether present frame surpasses the N frame; The space gaussian filtering result images of the original resolution obtained with step 1 if not,

as the image Q after denoising _i, enter step 9; If so, enter step 5;

Step 5: read present frame N frame adjacent image { P before _i-N, P _i-N+1..., P _i-1, and respectively with present frame space gaussian filtering result images P _idiffer from, obtain the difference diagram of N frame low resolution; In k frame difference figure, the difference value of arbitrfary point is designated as Δ p _ik=p _i-p _k, i means the sequence number of present frame, the sequence number that k is past frame, and k ∈ [i-N, i-N+1 ..., i-1], pi means the space gaussian filtering result images P of present frame low resolution _ithe pixel value of middle optional position, p _kspace gaussian filtering result images P for the low resolution of past frame _kthe pixel value of middle correspondence position;

Step 6: the difference diagram of N frame low resolution of take is input, asks over the weight Grt ' of N two field picture original resolution:

${\mathrm{Grt}}^{\′}\left({p^{\′}}_{\mathrm{ik}}\right)=\mathrm{\Γ}\left(\mathrm{Grt}\left({\mathrm{\Δp}}_{\mathrm{ik}}\right)\right)=\mathrm{\Γ}\left(\exp (-\frac{{{\mathrm{\Δp}}_{\mathrm{ik}}}^2}{{\mathrm{\σ}}^2})\right)$

In formula, Γ () means the weight map liter of low resolution is sampled as the weight map of original resolution, the down-sampled position corresponding relation with the low-resolution image pixel in original resolution image when down-sampled according to the step 1 average, while rising sampling, a point of the weight map of low resolution is corresponded to corresponding piece in the original resolution weight map, in this piece, the weighted value of each pixel is all got the weighted value of respective point in the weight map of low resolution; Grt ' (p ' _ik) be any pixel p ' of weight map of k frame original resolution _ikweight, Grt (Δ p _ik) be the weight of the weight map respective point of k frame low resolution; σ is customized parameter;

Step 7: the image { Q of reading over after the denoising of N frame _i-N, Q _i-N+1..., Q _i-1, in conjunction with current frame image Y _i, and the weight Grt ' of the past N two field picture that obtains of step 6, be weighted on average, tries to achieve the time-domain filtering result images

:

$q_t^i=\frac{y_i+\underset{k\∈[i-N,i-N+1,\·\·\·,i-1]}{\mathrm{\Σ}}{\mathrm{Grt}}^{\′}\left({p^{\′}}_{\mathrm{ik}}\right)\×q_k}{1+\underset{k\∈[i-N,i-N+1,\·\·\·,i-1]}{\mathrm{\Σ}}{\mathrm{Grt}}^{\′}\left({p^{\′}}_{\mathrm{ik}}\right)}$

In formula,

time-domain filtering result images for present frame

in any pixel value of pixel, y _ifor current frame image Y _ithe pixel value of middle respective pixel point, q _kfor the image Q after the denoising of past N frame _kthe pixel value of middle respective pixel point, k ∈ [i-N, i-N+1 ..., i-1];

Step 8: by the time-domain filtering result images that step 7 is obtained the space gaussian filtering result images of the original resolution obtained with step 1

be weighted on average, the filtering of trying to achieve present frame is Q as a result _i, the image as after denoising, enter step 9;

Step 9: the image Q after the storage denoising _i.

Further, in described step 8, average weighted concrete grammar is: with past N frame weight and w _grtfor basis, final denoising result is calculated as follows:

$q_i=\left\{\begin{array}{c}{q}_t^i,w_{\mathrm{Grt}}>\mathrm{thr}\\ \frac{w_{\mathrm{Grt}}}{\mathrm{thr}}{q}_t^i+\frac{(\mathrm{thr}-w_{\mathrm{Grt}})}{\mathrm{thr}}{q}_s^i,w_{\mathrm{Grt}}\≤\mathrm{thr}\end{array}\right.$

In formula, q _ifor the image Q after the denoising of present frame _iin any pixel value of pixel,

time-domain filtering result images for present frame

the pixel value of middle respective pixel point,

space gaussian filtering result images for the original resolution of present frame the pixel value of middle respective pixel point, past N frame weight and $w_{\mathrm{Grt}}=\underset{k\∈[i-N,i-N+1,\·\·\·,i-1]}{\mathrm{\Σ}}{\mathrm{Grt}}^{\′}\left({p^{\′}}_{\mathrm{ik}}\right),$ The threshold value of thr for setting, and 0≤thr≤N.

Further, in described step 1, the size of down-sampled is: when photosensitivity ISO400 and when following the size of down-sampled be 3 * 3; When ISO400-ISO1600, the size of down-sampled is 10 * 10; When ISO1600 and when above the size of down-sampled be 20 * 20.

Further, in described step 2, the window size of gaussian filtering is 5 * 5.

Further, in described step 3, the value of N is 4,6 or 8.

Further, in described step 6, the value of σ is 1-10.

Further, in described step 6, the value of thr is 0.6N.

The present invention has following technique effect compared with prior art:

1. the present invention is based on the average method of pixel value Similarity-Weighted and carry out video denoising, can guarantee the real-time of algorithm, and the finiteness of resource consumption, make and realize becoming possibility in digital imaging apparatus.

2. the present invention carries out the pre-filtering operation of the down-sampled and gaussian filtering of average by the image to the right to participate in re-computation, greatly suppressed the impact of noise on weight calculation, make weight calculation more accurate, can in the face of the super large noise time, correctly distinguish the variations such as noise and motion, illumination.

3. the present invention, by using and remove the image of making an uproar well in the past when the weighted average, has further improved denoising effect.Because make to spend the image of making an uproar, be equivalent to do being weighted average frame the pre-filtering of another way.Above-mentioned advantage 2 and this advantage have guaranteed algorithm good denoising effect in the face of the super large noise time.

4. changing obviously zone, add the space filtering result, made up the deficiency of time-domain filtering at region of variation.

In sum, the present invention adopts pre-filtering thought to improve the denoising effect of algorithm, particularly at the denoising effect when the super large noise, and real-time is good, computational resource consumption is few, can in hardware device, realize, be a kind ofly can efficient calculation can obtain the video denoising method of good result again.

The accompanying drawing explanation

Fig. 1 is schematic flow sheet of the present invention;

The schematic flow sheet (N=4) of Fig. 2 for meaning with multiframe;

Fig. 3 is pixel difference absolute value respective weights functional arrangement (σ=3);

Fig. 4 is the inventive method and additive method denoising result and original video on test video Salesman and adds video the 40th two field picture of making an uproar (noise criteria is poor is 100);

Fig. 5 is the inventive method and the test result figure of additive method on the real scene shooting video;

The hardware pictorial diagram of Fig. 6 for realizing by the present invention.

Embodiment

Below in conjunction with the drawings and specific embodiments, the invention will be further described.

As depicted in figs. 1 and 2, of the present invention towards the super large noise real-time video denoising method based on pre-filtering, comprise following steps:

Step 1: input current frame image Y _i, i means the sequence number of present frame, i=1,2,3 ..., current frame image is carried out to gaussian filtering, obtain the space gaussian filtering result images of original resolution

; Current frame image is done to average down-sampled simultaneously, obtain the low-resolution image of average after down-sampled; Described average is down-sampled is that current frame image is divided into to down-sampled of several formed objects, usings the mean value of all pixels in described down-sampled as this sampled value of down-sampled.

According to the difference that arranges of sensor photosensitive degree (ISO), the size of down-sampled is also different: when photosensitivity ISO400 and when following the size of down-sampled be 3 * 3; When ISO400-ISO1600, the size of down-sampled is 10 * 10; When ISO1600 and when above the size of down-sampled be 20 * 20.

Step 2: the low-resolution image after down-sampled to described average carries out gaussian filtering, obtains low-resolution spatial gaussian filtering result images P _i, i means the sequence number of present frame, i=1,2,3 ...The window size that carries out gaussian filtering in the present embodiment is 5 * 5.

Step 3: the low-resolution spatial gaussian filtering result images P of storage after step 2 is processed _i; Judge that whether present frame surpasses the N frame, when i>N, will be on the time when storage apart from present frame, a frame farthest substitutes, only storage comprises present frame P _iand the N frame adjacent image { P before present frame _i-N, P _i-N+1..., P _i-1; Described N is predefined value.The value of N is preferably 4,6 or 8.In the present embodiment, N is 4.

Here store low-resolution space gaussian filtering result images P _ito make poor needs during for the subsequent calculations weight.If store low-resolution space gaussian filtering result images P not here _i, calculate so each time weight and do when poor, all need frame to the past to carry out the down-sampled and gaussian filtering of average and process, wasted computational resource.

Step 4: judge whether present frame surpasses the N frame; The space gaussian filtering result images of the original resolution obtained with step 1 if not,

as the image Q after denoising _i, enter step 9; If so, enter step 5.

Step 5: read present frame N frame adjacent image { P before _i-N, P _i-N+1..., P _i-1, and respectively with present frame space gaussian filtering result images P _idiffer from, obtain the difference diagram of N frame low resolution; In k frame difference figure, the difference value of arbitrfary point is designated as Δ p _ik=p _i-p _k, i means the sequence number of present frame, the sequence number that k is past frame, and k ∈ [i-N, i-N+1 ..., i-1], p _ithe space gaussian filtering result images P that means the present frame low resolution _ithe pixel value of middle optional position, p _kspace gaussian filtering result images P for the low resolution of past frame _kthe pixel value of middle correspondence position.

Step 6: the difference diagram of N frame low resolution of take is input, asks over the weight Grt ' of N two field picture original resolution:

${\mathrm{Grt}}^{\&prime;}\left({p^{\&prime;}}_{\mathrm{ik}}\right)=\mathrm{\&Gamma;}\left(\mathrm{Grt}\left({\mathrm{\&Delta;p}}_{\mathrm{ik}}\right)\right)=\mathrm{\&Gamma;}\left(\exp (-\frac{{{\mathrm{\&Delta;p}}_{\mathrm{<figure-callout\; id="2"\; label="ik"\; filenames="HDA0000391813000000031.png"\; state="\{\{state\}\}">ik</figure-callout>}}}^2}{{\mathrm{\&sigma;}}^2})\right)$

In formula, Γ () means the weight map liter of low resolution is sampled as the weight map of original resolution, the down-sampled position corresponding relation with the low-resolution image pixel in original resolution image when down-sampled according to the step 1 average, while rising sampling, a point of the weight map of low resolution is corresponded to corresponding piece in the weight map of original resolution, in this piece, the weighted value of each pixel is all got the weighted value of respective point in the weight map of low resolution; Grt ' (p ' _ik) be any pixel p ' of weight map of k frame original resolution _ikweight, Grt (Δ p _ik) be the weight of the weight map respective point of k frame low resolution, and

σ is customized parameter.The value of σ is preferably 1-10, and in the present embodiment, the value of σ is 3.

The weight of each point in the weight map of low resolution in this step, corresponding to a piece of the weight map of original resolution.As shown in Figure 3, be the schematic diagram of weighting function curve in the present embodiment, abscissa means the pixel difference absolute value, ordinate means the weighted value obtained.Can find out, the result of weight calculation makes compares the point that value differences is large with present frame, and weight is little; The point that pixel value is similar, weight is large.

Step 7: the image { Q of reading over after the denoising of N frame _i-N, Q _i-N+1..., Q _i-1, in conjunction with current frame image Y _i, and the weight Grt ' of the past N two field picture original resolution that obtains of step 6, be weighted on average, tries to achieve the time-domain filtering result images

:

$q_t^i=\frac{y_i+\underset{k\&Element;[i-N,i-N+1,\&CenterDot;\&CenterDot;\&CenterDot;,i-1]}{\mathrm{\&Sigma;}}{\mathrm{Grt}}^{\&prime;}\left({p^{\&prime;}}_{\mathrm{ik}}\right)\&times;q_k}{1+\underset{k\&Element;[i-N,i-N+1,\&CenterDot;\&CenterDot;\&CenterDot;,i-1]}{\mathrm{\&Sigma;}}{\mathrm{Grt}}^{\&prime;}\left({p^{\&prime;}}_{\mathrm{ik}}\right)}$

In formula, time-domain filtering result images for present frame

in any pixel value of pixel, y _ifor current frame image Y _ithe pixel value of middle respective pixel point, q _kfor the image Q after the denoising of past N frame _kthe pixel value of middle respective pixel point, k ∈ [i-N, i-N+1 ..., i-1].

Step 8: by the time-domain filtering result images that step 7 is obtained

the space gaussian filtering result images of the original resolution obtained with step 1

be weighted on average, the filtering of trying to achieve present frame is Q as a result _i, the image as after denoising, enter step 9.

In this step, in conjunction with time-domain filtering result and space gaussian filtering result, more obvious according to variations such as motion, illumination, the space filtering weight is larger; Otherwise more not obvious, the time-domain filtering weight is weighted more greatly on average, tries to achieve last filtering result.

Step 9: the image Q after the storage denoising _i.

The purpose of the image of storage after denoising is can directly take out while being convenient to calculate next time, in the present embodiment when storage, the time on apart from present frame after a frame denoising farthest image replaced, i.e. the image Q after the denoising of preservation present frame in memory space only _iand the image { Q after N frame denoising in the past _i-N, Q _i-N+1..., Q _i-1.

The present invention is based on the image pre-filtering video is carried out to real-time de-noising.In step 1 and step 2, adopt respectively the down-sampled and gaussian filtering of average to carry out pre-filtering to original image, step 1 and step 2 are equivalent to former figure has been done a large weighted average and then carried out down-sampled.Adopting step 1 and step 2 to replace large average weighted purpose is in order to save computational resource.For example, if do the gaussian filtering of a 50x50, calculate so 2500 multiplication of some needs and addition, and buffer memory 50 row view data, very consumption of natural resource.Adopt the down-sampled mode of average, use a register just can complete the cumulative of a piece, saved the data line of multiplying and buffer memory, greatly saved resource.And down-sampled sacrifice is to replace the weight of a piece with the weight of a point, make the weight at details place meticulous not.So, in practical operation, down-sampled yardstick can not be too large.The present invention adopts down-sampled mode of being combined with gaussian filtering.That is, the filtering of 50x50 is split as the filtering of the down-sampled and 5x5 of the average of 10x10.The filtering that realizes a 5x5 in hardware is a very simple thing.In specific implementation, also need the difference (difference of transducer ISO) according to noise level that down-sampled different scale is set.

Although the image after pre-filtering is the little figure after down-sampled here, and thickens, it has suppressed noise greatly, makes the calculating of weight become very accurate.And the image of down-sampled filtering is only for the weight of the weights figure each point that calculates low resolution, further obtain the weights of each pixel in the original resolution weight map by it, be used for being weighted the average original resolution image that remains in step 7, can't affect resolution and the definition of last denoising result.In addition, in order further to improve denoising effect, the present invention takes full advantage of the result after the past frame denoising.In step 7, during weighted average, past frame is used the image after denoising, and this is equivalent in fact to done a strong pre-filtering for average weighted frame.Weighted average comprises weight and two aspects of the value of being weighted, and the present invention is directed to two aspects and has carried out the pre-filtering of different modes, makes the method still have good denoising effect in the face of the super large noise time.

From the angle analysis of real-time, the computing the present invention relates to is very simple.So that certain any processing is considered, in the present embodiment except the down-sampled cumulative and 5x5 gaussian filtering of average, also need to take out the point of same position in the down-sampled filtering figure of N frame, do N subtraction, then try to achieve by table look-up (in advance weight is asked for to function and be designed to look-up table stores) weight that the pixel difference value is corresponding, last weighted average is tried to achieve the time filtering result.For time filtering, only need subtraction N time, table look-up for N time, N multiplication, N sub-addition and 1 division.Because we generally do frame when accumulation, N=4,6, so or 8 calculate a point and only need very limited computational resource.And, in sensor-based digital imaging apparatus, view data is that the mode with pointwise line by line obtains, this pipeline characteristics, make algorithm of the present invention can accomplish real-time implementation fully.

Finally, due to changing large zone because the weight of past frame is little, substantially can only take present frame as main, make the noise remove degree inadequate.Therefore need to be in conjunction with space filtering result, i.e. step 8.

In step 8, average weighted concrete grammar is: take past N frame weight and wGrt as basis, final denoising result is calculated as follows:

$q_i=\left\{\begin{array}{c}{q}_t^i,w_{\mathrm{Grt}}>\mathrm{thr}\\ \frac{w_{\mathrm{Grt}}}{\mathrm{thr}}{q}_t^i+\frac{(\mathrm{thr}-w_{\mathrm{Grt}})}{\mathrm{thr}}{q}_s^i,w_{\mathrm{Grt}}\&le;\mathrm{thr}\end{array}\right.$

In formula, q _ifor the image Q after the denoising of present frame _iin any pixel value of pixel,

time-domain filtering result images for present frame

the pixel value of middle respective pixel point, space gaussian filtering result images for the original resolution of present frame

the pixel value of middle respective pixel point, past N frame weight and the threshold value of thr for setting, and 0≤thr≤N.The value of thr is preferably 0.6N, and in the present embodiment, the value of thr is 2.4.

In step 8, time-domain filtering result and airspace filter result have been carried out to weighted average, the present frame weight is made as 1, supposes that past frame and present frame are identical, and weight Grt ' is also 1.Even past frame and present frame are identical, so the cumulative time domain mean filter that is equivalent to of each frame, therefore past N frame weight and w _grtmaximum be N.And if current point is region of variation, so its weight Grt ' (p ' _ik) will be less than 1, past N frame weight and w _grtto be less than N, and w _grtless, mean that the possibility that current point is region of variation is larger.

Denoising effect of the present invention can further illustrate by following emulation experiment.

One, emulation experiment one

1. experiment condition and content

The experiment simulation environment is: MATLAB R2009a, CPU Intel Core i3-21003.10GHz, internal memory 1.0GB, Windows XP Professional.

Experiment content comprises: use the video sequence Salesman of 288x352, and Bridge-close, and Paris, adding the white Gaussian noise standard deviation is respectively 5,15,30,50,75,100.Exemplary process VBM3D and the SURE-LET of two kinds of current denoising effect the bests introducing in the background technology selected in control methods.Evaluation index has been selected Y-PSNR PSNR, with structural similarity sex index SSIM.

2. experimental result

Fig. 4 is the inventive method and additive method denoising result and original video on test video Salesman and adds video the 40th two field picture of making an uproar, and the noise criteria of interpolation is poor is 100, simulation super large noise situations.From experimental result picture, can find out, the whole bag of tricks all can not the former figure of complete recovery.But examine and, in background area, the effect of algorithm of the present invention is best, the profile of books for example, the present invention does not have ambiguity to produce.Because background is stagnant zone, only have the frame accumulation results of time domain, its denoising effect is best.And in moving region, as the hand of foreground people, the hand of motion that each method is all fuzzy, which is better and which is worse to be difficult to judgement.Generally speaking, the present invention, in this experiment contrast, has obtained with the current best approach and can match in excellence or beauty, the effect of even having omited.

Y-PSNR PSNR and structural similarity sex index SSIM experimental result contrast table, in Table 1, provided the experimental result under different noises for each video.

Table 1 experimental result

As can be seen from Table 1, although the inventive method at noise hour, advantage is also not obvious, when noise is larger, can obtain and SURE-LET, and VBM3D compares favourably, even better effect.For transducer, when illumination is strong, noise is little, and denoising promotes limited to picture quality; When illumination is weak, noise is large, and it is particularly important that the effect of denoising just seems.And method of the present invention can, under the large noise situations of low-light (level), obviously promote denoising effect.

Two, emulation experiment two

Because actual imageing sensor noise not only shows as white Gaussian noise, but the noise mixed by various noises such as impulsive noise, quantizing noise, electronic noises.For a nearlyer step illustrates real work effect of the present invention, this emulation experiment has gathered noisy image and has been tested from actual imageing sensor.

Fig. 5 is the inventive method and the test result figure of additive method on the real scene shooting video.As can be seen from Figure 5, under the large noise situations of actual low-light (level), the inventive method is still worked well, can obviously suppress noise, makes background detail to display, and has obtained best denoising effect.Other two kinds of method denoising effects obviously reduce.

In sum, the present invention is a kind of video denoising method average based on the pixel value Similarity-Weighted.Compare with conventional method, introduced pre-filtering thought.Carry out the down-sampled pre-filtering with gaussian filtering of average by the image to the right to participate in re-computation and operate, greatly suppressed the impact of noise on weight calculation, make weight calculation more accurate.In addition, to using past frame when the weighted average, with the image after denoising before this, replace original image, be equivalent to do participating in average weighted frame the pre-filtering of another kind of mode.The pre-filtering of two aspects makes the present invention obtain good effect on time domain, particularly, when the super large noise in the face of under low-light (level), still can obtain denoising effect preferably.Owing to not having past frame information therefore to have added the space filtering result for accumulation, made up the deficiency of time-domain filtering at region of variation in the zone of motion illumination variation.Aspect real-time, each step of algorithm does not relate to complex calculation, and the down-sampled use accumulator of average can complete, and gaussian filtering also is very easy to, and the weighted average of N frame only relates to N multiplication and addition, and a division.That calculates is simple, guaranteed that algorithm can accomplish real-time implementation, and resource consumption is limited, makes the present invention can be embedded in digital imaging apparatus, and the hardware material object after realization as shown in Figure 6.Wherein kernel processor chip has adopted fpga chip.

Claims (7)

1. One kind towards the super large noise real-time video denoising method based on pre-filtering, it is characterized in that comprising the following steps:

   Step 1: input current frame image Y _i, i means the sequence number of present frame, i=1,2,3 ..., current frame image is carried out to gaussian filtering, obtain the space gaussian filtering result images of original resolution ; Current frame image is done to average down-sampled simultaneously, obtain the low-resolution image of average after down-sampled; Described average is down-sampled is that current frame image is divided into to down-sampled of several formed objects, usings the mean value of all pixels in described down-sampled as this sampled value of down-sampled;

   Step 2: the low-resolution image after down-sampled to described average carries out gaussian filtering, obtains low-resolution spatial gaussian filtering result images P _i, i means the sequence number of present frame, i=1,2,3

   Step 3: the low-resolution spatial gaussian filtering result images P of storage after step 2 is processed _i; Judge that whether present frame surpasses the N frame, when i>N, will be on the time when storage apart from present frame, a frame farthest substitutes, only storage comprises present frame P _iand the N frame adjacent image { P before present frame _i-N, P _i-N+1..., P _i-1; Described N is predefined value;

   Step 4: judge whether present frame surpasses the N frame; The space gaussian filtering result images of the original resolution obtained with step 1 if not,

   

   as the image Q after denoising _i, enter step 9; If so, enter step 5;

   Step 5: read present frame N frame adjacent image { P before _i-N, P _i-N+1..., P _i-1, and respectively with present frame space gaussian filtering result images P _idiffer from, obtain the difference diagram of N frame low resolution; In k frame difference figure, the difference value of arbitrfary point is designated as Δ p _ik=p _i-p _k, i means the sequence number of present frame, the sequence number that k is past frame, and k ∈ [i-N, i-N+1 ..., i-1], p _ithe space gaussian filtering result images P that means the present frame low resolution _ithe pixel value of middle optional position, p _kspace gaussian filtering result images P for the low resolution of past frame _kthe pixel value of middle correspondence position;

   Step 6: the difference diagram of N frame low resolution of take is input, asks over the weight Grt ' of N two field picture original resolution:

   ${\mathrm{Grt}}^{\&prime;}\left({p^{\&prime;}}_{\mathrm{ik}}\right)=\mathrm{\&Gamma;}\left(\mathrm{Grt}\left({\mathrm{\&Delta;p}}_{\mathrm{ik}}\right)\right)=\mathrm{\&Gamma;}\left(\exp (-\frac{{{\mathrm{\&Delta;p}}_{\mathrm{ik}}}^2}{{\mathrm{\&sigma;}}^2})\right)$

   In formula, Γ () means the weight map liter of low resolution is sampled as the weight map of original resolution, the down-sampled position corresponding relation with the low-resolution image pixel in original resolution image when down-sampled according to the step 1 average, while rising sampling, a point of the weight map of low resolution is corresponded to corresponding piece in the original resolution weight map, in this piece, the weighted value of each pixel is all got the weighted value of respective point in the weight map of low resolution; Grt ' (p ' _ik) be any pixel p ' of weight map of k frame original resolution _ikweight, Grt (Δ p _ik) be the weight of the weight map respective point of k frame low resolution; σ is customized parameter;

   Step 7: the image { Q of reading over after the denoising of N frame _i-N, Q _i-N+1..., Q _i-1, in conjunction with current frame image Y _i, and the weight Grt ' of the past N two field picture original resolution that obtains of step 6, be weighted on average, tries to achieve the time-domain filtering result images

   

   $q_t^i=\frac{y_i+\underset{k\&Element;[i-N,i-N+1,\&CenterDot;\&CenterDot;\&CenterDot;,i-1]}{\mathrm{\&Sigma;}}{\mathrm{Grt}}^{\&prime;}\left({p^{\&prime;}}_{\mathrm{ik}}\right)\&times;q_k}{1+\underset{k\&Element;[i-N,i-N+1,\&CenterDot;\&CenterDot;\&CenterDot;,i-1]}{\mathrm{\&Sigma;}}{\mathrm{Grt}}^{\&prime;}\left({p^{\&prime;}}_{\mathrm{ik}}\right)}$

   In formula,

   

   time-domain filtering result images for present frame

   

   in any pixel value of pixel, y _ifor current frame image Y _ithe pixel value of middle respective pixel point, q _kfor the image Q after the denoising of past N frame _kthe pixel value of middle respective pixel point, k ∈ [i-N, i-N+1 ..., i-1];

   Step 8: by the time-domain filtering result images that step 7 is obtained

   

   the space gaussian filtering result images of the original resolution obtained with step 1

   

   be weighted on average, the filtering of trying to achieve present frame is Q as a result _i, the image as after denoising, enter step 9;

   Step 9: the image Q after the storage denoising _i.
2. According to claim 1 towards the super large noise real-time video denoising method based on pre-filtering, it is characterized in that: in described step 8, average weighted concrete grammar is: with past N frame weight and w _grtfor basis, final denoising result is calculated as follows:

   $q_i=\left\{\begin{array}{c}{q}_t^i,w_{\mathrm{Grt}}>\mathrm{thr}\\ \frac{w_{\mathrm{Grt}}}{\mathrm{thr}}{q}_t^i+\frac{(\mathrm{thr}-w_{\mathrm{Grt}})}{\mathrm{thr}}{q}_s^i,w_{\mathrm{Grt}}\&le;\mathrm{thr}\end{array}\right.$

   In formula, q _ifor the image Q after the denoising of present frame _iin any pixel value of pixel, time-domain filtering result images for present frame

   

   the pixel value of middle respective pixel point,

   

   space gaussian filtering result images for the original resolution of present frame the pixel value of middle respective pixel point, past N frame weight and $w_{\mathrm{Grt}}=\underset{k\&Element;[i-N,i-N+1,\&CenterDot;\&CenterDot;\&CenterDot;,i-1]}{\mathrm{\&Sigma;}}{\mathrm{Grt}}^{\&prime;}\left({p^{\&prime;}}_{\mathrm{ik}}\right),$ The threshold value of thr for setting, and 0≤thr≤N.
3. According to claim 1 towards the super large noise real-time video denoising method based on pre-filtering, it is characterized in that: in described step 1, the size of down-sampled is: when photosensitivity ISO400 and when following the size of down-sampled be 3 * 3; When ISO400-ISO1600, the size of down-sampled is 10 * 10; When ISO1600 and when above the size of down-sampled be 20 * 20.
4. According to claim 1 towards the super large noise real-time video denoising method based on pre-filtering, it is characterized in that: in described step 2, the window size of gaussian filtering is 5 * 5.
5. According to claim 1 towards the super large noise real-time video denoising method based on pre-filtering, it is characterized in that: in described step 3, the value of N is 4,6 or 8.
6. According to claim 1 towards the super large noise real-time video denoising method based on pre-filtering, it is characterized in that: in described step 6, the value of σ is 1-10.
7. According to claim 2 towards the super large noise real-time video denoising method based on pre-filtering, it is characterized in that: in described step 8, the value of thr is 0.6N.

Images