CN100502466C - Method and system for video data real-time de-noising - Google Patents

Abstract

The present invention provides a real time denoising method for a video data, which comprises the following procedures: estimation for the noise energy; preprocessing filtration for the current frame image; movement detection; background model renewal according to the piece or pixel which has a stationary property detected. The background model comprises a reference layer and at least one temporary layer, the reference layer is used to store the main reference data, and the temporary layer is used to store assistant reference data. The renewal of the reference layer should utilize the relative assistant reference data in the temporary layer. If the property of the target piece or pixel is stationary, and the relative changing level of the reference data accord with the preset condition, then the filtration should be executed according to the reference data provided by the background model, otherwise, the normal filtration should be executed. The utility model introduces the background mode into the video denoising for the first time, and brings a special background model suitable for the video denoising, which can satisfy the demand of the high intelligibility video denoising in real time.

Description

A kind of method and system of video data real-time de-noising

Technical field

The present invention relates to the video data process field, particularly relate to a kind of method and system of video data real-time de-noising.

Background technology

Along with the development of computer and digital camera technology, the life that affects people that video data is just more and more deep, along with people are more and more higher to the requirement of video data, at the processing method of video data denoising also in continuous development.

Traditional video Denoising Algorithm can comprise airspace filter and time-domain filtering.Wherein, simple airspace filter method, because it does not consider time-domain information, and there is randomness in the noise of same position between the video sequence frame, therefore causes the tangible scintillation between the filtered consecutive frame easily; And, then can cause that it fails to match or error owing to the existence of motion for simple time-domain filtering technology, residual or " ghost " phenomenon of noise appears.Therefore, time-space domain filtering technique more preferably in the prior art, with respect to the algorithm in other territory, its amount of calculation is a little more than spatial domain and time-domain filtering, but effect obviously is better than spatial domain and time-domain filtering, thus its can be applied to high definition, the real-time video denoising.

In the filtering algorithm of time-space domain, general normal employing motion detection scheme and motion estimation scheme.They can both be distinguished dynamic area and static region, and adopt different filtering methods to handle at zones of different.By contrast, estimation can provide more reference data for filtering than motion detection, and therefore the filtering algorithm based on estimation can access better filter effect.But only with reference to the data of adjacent several frames, promptly its time domain filtering performance is general, so they all can not eliminate residual or " ghost " phenomenon of noise of interframe well for above two kinds of algorithms.

Therefore, need the urgent technical problem that solves of those skilled in the art to be exactly:, to propose a kind of technical scheme that can obtain the video denoising of better filter effect at the demand of high definition, real-time video denoising.

Summary of the invention

Technical problem to be solved by this invention provides a kind of method and system of video data real-time de-noising, can satisfy the demand of video denoising high definition, real-time, and obtains than the better filter effect of prior art.

In order to address the above problem, the invention discloses a kind of method of video data real-time de-noising, specifically can may further comprise the steps: image data, carry out noise energy and estimate the output noise extent index; According to described noise level parameter, finish preliminary treatment filtering at current frame image; At pretreated image, carry out motion detection; The result of described motion detection is carried out the attributive classification of each piece, and be static piece or pixel, carry out the renewal of background model at attribute; Described background model comprises a reference layer and at least one temporary layer, and described reference layer is used to store main reference data, and described temporary layer is used to store the auxiliary reference data; The renewal of described reference layer need be used corresponding auxiliary reference data in the described temporary layer; If the attribute of object block or pixel is static, and the intensity of variation of its corresponding reference layer data conforms to a predetermined condition, and then the reference data that is provided according to the background model reference layer is carried out filtering; Otherwise, carry out common filtering.

Preferably, described is static piece or pixel at attribute, can finish the renewal of background model by following steps:, then upgrade the related data in its affiliated background model if the data of current block or pixel have identical time domain statistical property with corresponding reference layer data; Otherwise, with in the existing background model effectively the corresponding data in the temporary layer of reference value minimum abandon, with the data of current block or pixel data as this temporary layer.

Preferably, finish after the renewal of the related data in the background model under current block or the pixel, can also comprise: when the steadiness parameter of this piece or pixel conforms to a predetermined condition, then the related data with the temporary layer of this piece or pixel places reference layer.

Wherein, the mode of described common filtering can comprise: any in linear low-pass filtering algorithm, non-linear sort algorithm or the 3D-DUNCAN filtering algorithm.

According to another embodiment of the present invention, a kind of system of video data real-time de-noising is disclosed, comprising:

The Noise Estimation module is used for image data, carries out noise energy and estimates the output noise extent index;

The preliminary treatment filtration module is used for finishing preliminary treatment filtering according to described noise level parameter at current frame image;

The piece motion detection block is used for carrying out the piece motion detection at pretreated image;

The block sort module is used for the result of described motion detection is carried out the attributive classification of each piece;

The background model update module, being used at attribute is static piece or pixel, carries out the renewal of background model; Described background model comprises a reference layer and at least one temporary layer, and described reference layer is used to store main reference data, and described temporary layer is used to store the auxiliary reference data; The renewal of described reference layer need be used corresponding auxiliary reference data in the described temporary layer;

The post processing and filtering module, the attribute that is used for when object block or pixel is static, and the intensity of variation of its corresponding reference layer data carries out filtering according to the reference data that the background model reference layer is provided when conforming to a predetermined condition; Otherwise, carry out common filtering.

Preferably, described background model update module may further include:

Judge submodule, be used to judge whether the data of current block or pixel have identical time domain statistical property with corresponding reference layer data; If, then start first updating submodule, otherwise, second updating submodule started;

First updating submodule is used for adding up the related data of upgrading its affiliated background model;

Second updating submodule, be used for existing background model effectively the corresponding data in the temporary layer of reference value minimum abandon, with the data of current block or pixel data as this temporary layer.

Preferably, described background model update module can also comprise: data advance submodule, be used for after the renewal of related data of the background model under finishing current block or pixel, when the steadiness parameter of this piece or pixel conformed to a predetermined condition, then the related data with the temporary layer of this piece or pixel placed reference layer.

Wherein, the mode of described common filtering can comprise: any in linear low-pass filtering algorithm, non-linear sort algorithm or the 3D-DUNCAN filtering algorithm.

According to another embodiment of the present invention, a kind of method of optimizing background model is disclosed, described background model comprises a reference layer and at least one temporary layer, and described reference layer is used to store main reference data, and described temporary layer is used to store the auxiliary reference data; The renewal of described reference layer need be used corresponding auxiliary reference data in the described temporary layer; Described method comprises: the attributive classification that carries out each piece at the result of motion detection, and be static piece or pixel at attribute in the present frame, judge whether the data of current block or pixel have identical time domain statistical property with corresponding reference layer data; If then statistics is upgraded the related data in its affiliated background model; Otherwise, with in the existing background model effectively the corresponding data in the temporary layer of reference value minimum abandon, with the data of current block or pixel data as this temporary layer.

Preferably, finish after the renewal of the related data in the background model under current block or the pixel, also comprise: when the steadiness parameter of this piece or pixel conforms to a predetermined condition, then the related data with the temporary layer of this piece or pixel places reference layer.

Compared with prior art, the present invention has the following advantages:

With respect to the common time-space domain noise-removed technology scheme based on motion detection or estimation, the present invention is incorporated into the notion of background model in the video denoising first; And because background model of the present invention is according to setting up on the multiframe period of the day from 11 p.m. to 1 a.m territory, if its stabilization time is of a specified duration more, the time domain of its background model can be with reference to just approaching more zero noise model of property.And general frame filter, only may be with reference to the data of adjacent several frames.Therefore, by contrast, the establishment of background model just means the statistical property of having utilized time domain more fully, that is to say that the video noise-removed technology based on background model of the proposition of institute of the present invention novelty can obtain more excellent time domain denoising performance.

Secondly, owing to background model common in the prior art is that identification and tracking at moving object proposes, they do not consider whether suppress noise on time domain, whether real-time adaptive changes, and only needs the identifiability between consideration present frame moving object and the background.Since the stability that the existence of noise can have influence on background model with can be with reference to property, therefore background model in the past all is to be based upon under the prerequisite that small noise disturbs.And at the real-time video denoising, background model must be utilized the time statistical property, suppresses noise between frames, and keeps certain stability, and it can not exist too big time delay and too big amount of calculation in addition.Therefore, the present invention for better realize at high definition, the real-time video denoising, a kind of novel multilayer adaptive background model has been proposed, setting by the multilayer background model, keep the time domain statistics that follow-up filtering is had reference significance as much as possible, thereby realized denoising performance efficiently.

Video noise-removed technology scheme disclosed in this invention provides the background model of a function admirable, can take into account to have the video sequence of stablizing the video sequence of background and having astable background, goes for the video sequence of various noise intensities.But in sum, can learn, be the suitable environment of the best of the present invention at there being the video sequence of stablizing background.

Description of drawings

Fig. 1 is the schematic flow diagram of a kind of video data real-time de-noising method of the present invention embodiment;

Fig. 2 is a detailed step flow chart embodiment illustrated in fig. 1;

Fig. 3 is the structural representation of a kind of background model of the present invention;

Fig. 4 is the schematic flow diagram of the more new technological process of a kind of background model of the present invention;

Fig. 5 is the reprocessing schematic flow sheet that adopts the preferred embodiment of 3D-DUNCAN algorithm filtering;

Fig. 6 is the schematic diagram that data are selected when adopting the filtering of 3D-DUNCAN algorithm;

Fig. 7 is a kind of flow chart of steps of optimizing the method embodiment of background model of the present invention;

Fig. 8 is the structured flowchart of a kind of video data real-time de-noising system embodiment of the present invention.

Embodiment

For above-mentioned purpose of the present invention, feature and advantage can be become apparent more, the present invention is further detailed explanation below in conjunction with the drawings and specific embodiments.

Core concept of the present invention is: first background model is incorporated in the video denoising, and has proposed a kind of background model that is applicable to the video denoising of uniqueness.Background model in the past, its purpose are to detect moving object from background.Background model in this algorithm, its purpose in contrast, the time domain statistics after moving object stops just to have reference significance to the video denoising.Because the complexity and the unsteadiness of background, therefore, in order to obtain being fit to the background model of video denoising, and can provide good time domain reference data for the video denoising in general Denoising Algorithm; The major issue that the present invention need overcome just is, how to keep and upgrades the data that to subsequent video denoising has reference significance most.

The present invention can be used in numerous general or special purpose computingasystem environment or the configuration.For example: personal computer, server computer, handheld device or portable set, plate equipment, multicomputer system, the system based on microprocessor, set top box, programmable consumer-elcetronics devices, network PC, minicom, mainframe computer, comprise distributed computing environment (DCE) of above any system or equipment or the like.

The present invention can describe in the general context of the computer executable instructions of being carried out by computer, for example program module.Usually, program module comprises the routine carrying out particular task or realize particular abstract, program, object, assembly, data structure or the like.Also can in distributed computing environment (DCE), put into practice the present invention, in these distributed computing environment (DCE), by by communication network connected teleprocessing equipment execute the task.In distributed computing environment (DCE), program module can be arranged in the local and remote computer-readable storage medium that comprises memory device.

See figures.1.and.2, show the flow chart of steps and the schematic diagram of a kind of video data real-time de-noising method of the present invention, specifically can may further comprise the steps:

Step 101, image data are carried out noise energy and are estimated the output noise extent index.

In one embodiment of the invention, can all carry out the collection of data and the estimation of noise energy at the image of every frame.Certainly, in order to raise the efficiency, and because noise generally all has continuation, so in the preferred embodiment of the present invention, can finish data acquisition and noise energy and estimate to get final product every certain time interval.

In the prior art, digital collection equipment generally all can adopt the adapting to image gain techniques, and making can both provide brightness moderate image under high or low brightness situation.For example, when the collection scene was dark, image gain can become greatly, made image be amplified to and was fit to the degree that human eye is accepted.Noise level also can become along with image gain greatly, thereby causes the noise pollution of image also to aggravate but meanwhile.

Noise Estimation is an important step of noise adaptive-filtering, and its accuracy will have influence on follow-up performance in steps.Preferably, Noise Estimation can once get final product carrying out after the regular hour section, and resulting noise level parameter is all effective in this time period, and the concrete time interval is by the noise stability decision at scene.For Noise Estimation the concrete technical scheme that can adopt, disclose a variety ofly in the prior art, provide two kinds of optional Noise Estimation modes below, to be applicable to different application scenarioss.

Mode 1

Can adopt the noise level method of estimation based on flat site, the prerequisite of this technical scheme is to search out flat site.The searching of flat site is by finishing based on the texture ranking score parser on the Noise Estimation of certain levels of precision, perhaps being based upon recursiveness and seeking on the basis of regional Minimum Mean Square Error.Because the identification of flat site is to finish by follow-up block sort, so this moment, Noise Estimation may not be first step in total algorithm, but last step.The noise level parameter can be predicted according to the average variance of flat site and obtain.

Mode 2

Can adopt based on the noise level in " concealed wire " zone and estimate." concealed wire " zone is in pickup area together with image-region.Compare with image-region, " concealed wire " zone has the identical time for exposure, has identical exposure gain, but because very low and very even by the coverage light luminance that it received.Therefore " concealed wire " zone has the noise jamming characteristic identical with image-region.

General, according to smooth or " concealed wire " zone, the estimation of noise level can be obtained by following formula:

${\mathrm{\σ}}_t=\sqrt{\frac{1}{N-1}\underset{(x,y,t)\∈R}{\mathrm{\Σ}}{(f(x,y,t)-\mathrm{average})}^2}$

Wherein, N is illustrated in the number of the point that collects in the interval R; Average represents the mean value that this is interval; σ _tExpression t noise level parameter constantly, x, y represent interval coordinate, t express time, f (x, y, t) expression measured value.

Step 102, the described noise level parameter of foundation are finished preliminary treatment filtering at current frame image.

Preliminary treatment filtering algorithm in this step can adopt multiple feasible prior art.For example, Wiener filtering, linear low-pass filtering algorithm (as Kalman filtering, FIR low pass etc.), non-linear sort algorithm (as medium filtering etc.) etc.Below Wiener filtering is simply introduced:

Wei Na (Wiener) filtering is a kind of noise and fuzzy effective means that is usually used in removing degraded signal.It is to utilize the correlation properties of stationary random process and spectral characteristic is carried out filtering to the signal that is mixed with noise a kind of method.The two-dimentional expression formula of Wiener filtering can be expressed as follows:

$\hat{x}(\mathrm{<figure-callout\; id="1"\; label="n"\; filenames="C200710145743E00232.png"\; state="\{\{state\}\}">n</figure-callout>}1,n2)=\frac{{\mathrm{\&delta;}}_x^2}{{\mathrm{\&delta;}}_x^2+{\mathrm{\&delta;}}_{\mathrm{\&omega;}}^2}[y(\mathrm{<figure-callout\; id="1"\; label="n"\; filenames="C200710145743E00232.png"\; state="\{\{state\}\}">n</figure-callout>}1,n2)-{\mathrm{\&mu;}}_y]+{\mathrm{\&mu;}}_x---\left(1\right)$

Wherein, x (n1, n2) expression source image data, y (n1, n2) expression measured value, δ _ωThe variance (being the σ that obtains in the Noise Estimation) of expression noise ω, δ _xThe variance of expression source signal x, δ _yThe variance of expression measured value y, μ _xThe average of expression source data in adjacent domain, μ _yThe average of expression measured value in adjacent domain.Since in Video processing, μ _xWith δ _xUnknowable.At this moment, suppose that noise is that average is 0 additivity random noise, so μ _x=μ _y,

Then can be used as

Valuation, thereby can pass through δ _ωAnd δ _yCalculate δ _x

Then, the two-dimentional expression formula of Wiener filtering can be rewritten as follows:

$\hat{x}(\mathrm{<figure-callout\; id="1"\; label="n"\; filenames="C200710145743E00232.png"\; state="\{\{state\}\}">n</figure-callout>}1,n2)=\frac{{\mathrm{\&delta;}}_x^2}{{\mathrm{\&delta;}}_y^2}[y(\mathrm{<figure-callout\; id="1"\; label="n"\; filenames="C200710145743E00232.png"\; state="\{\{state\}\}">n</figure-callout>}1,n2)-{\mathrm{\&mu;}}_y]+{\mathrm{\&mu;}}_y$

By ${\mathrm{\&delta;}}_x^2>0,$ As can be known ${\mathrm{\&delta;}}_x^2=\left\{\begin{array}{c}{\mathrm{\&delta;}}_y^2-{\mathrm{\&delta;}}_{\mathrm{\&omega;}}^2,\mathrm{if}{\mathrm{\&delta;}}_y^2>{\mathrm{\&delta;}}_{\mathrm{\&omega;}}^2\\ 0,\mathrm{if}{\mathrm{\&delta;}}_y^2<={\mathrm{\&delta;}}_{\mathrm{\&omega;}}^2\end{array}\right.$

Can draw by above expression formula, the noise smoothing ability of filter with

With

Ratio direct relation is arranged.This ratio is reflecting the noise power of each point in the input signal power spectrum and the relativity of signal power indirectly.Therefore, Wiener filtering has the ability of adaptive-filtering to noise and image detail, is that the present invention recommends one of filters solutions of using.

, can introduce background model it is further processed through after the preliminary preliminary treatment current frame image.

The background model of setting up scene is first stage of sequence image motion analysis, also is one of core part in the motion analysis.Through years of researches, the researcher has proposed many moving object extraction algorithms at the special scenes sequence image both at home and abroad.For example more typical algorithm time series difference (Temporal Difference), background subtract the extracting method of method (Background Subtraction) and these three traditional moving objects of optical flow method (Opticalflow).Recently many moving object extraction algorithms all are to derive on the basis of three kinds of traditional algorithms and come.

The time difference method is to adopt between two or three consecutive frames based on the time difference of pixel and thresholding to extract moving region in the image in the continuous images sequence.For example Lipton etc. utilizes two frame difference methods to detect moving target from real video, and then is used for the classification and the tracking of target; An improved method is to utilize three-frame difference to replace two frame differences, has developed the hybrid algorithm that a kind of adaptive background subduction combines with three-frame difference as VSAM, and it can fast and effeciently detect moving target from background.

The background subtraction method is a kind of method the most frequently used in the present motion segmentation, and it is to utilize the difference of present image and background image to detect a kind of technology of moving region.It generally can provide characteristic the most completely, but for the variation of dynamic scene, and is responsive especially as the interference of illumination and external extraneous events etc.The simplest background model is the time average image, and most researcher is devoted to develop different background models at present, in the hope of reducing the influence of dynamic scene variation for motion segmentation.For example utilization such as Haritaoglu is minimum, maximum intensity value and maximum time difference value carry out statistical modeling for each pixel in the scene, and carry out periodically context update.

Motion detection based on optical flow approach has adopted the time dependent light stream characteristic of moving target, as Meyer etc. by coming the track algorithm of initialization in displacement calculating optical flow vector field based on profile, thereby extract effectively and the pursuit movement target.The advantage of this method is also can detect independently moving target under the prerequisite that camera motion exists.Yet most optical flow computation method is quite complicated, and noiseproof feature is poor, if there is not special hardware unit can not be applied to the real-time processing of full frame video stream.

But, background model in the past be mostly be used for that moving object detects, identification with follow the trail of, so this class background model is indifferent to the multiframe statistical property of time domain, only needs the identifiability between consideration present frame moving object and the background.For the video denoising, the purpose of background model is that the reference information with noise inhibiting ability is provided as much as possible, rather than detects moving object from background.Therefore, should keep the data that have the time statistical property most in the background model.If a moving object stops, for the video denoising, real background needs to keep, but does not exist in all senses with regard to present filtering so.Opposite with background model in the past, the time domain statistics after moving object stops just to have reference significance to the video denoising.Because the complexity and the unsteadiness of background, just be at a problem of the background model of video denoising how to keep and upgrade the data that to subsequent video denoising has reference significance most.

Based on above some consideration, this paper has proposed a kind of novel multilayer adaptive background model, and is introduced in the video Denoising Algorithm of time-space domain.This background model is not only applicable to static background, also is applicable to the background that polymorphism changes.In addition, after background changed, it can finish the correction of background model in the of short duration time.

Step 103, at pretreated image, carry out motion detection;

Preferably, the piece motion detection in the step 103 can be with reference to the time series difference algorithm based on continuous three frames.In natural scene, usually there are some small variable domains, for example, the slight fluctuation etc. of swing, lake surface gently of leaf. at these characteristics, based on the time series difference algorithm of continuous three frames can avoid since camera slightly rocks and background in the interference that the moving region is detected of light exercises such as leaf.Suppose that video sequence derives from the fixed point camera this moment.According to three frame difference algorithms,, think that then moving appears in current pixel point if the gray value of a pixel between present frame and former frame, and between present frame and forward direction second frame, marked change has taken place.

For example, adopt block-based three frame SAD algorithms in the present embodiment, that is,, think that then there is motion in current block if following expression is set up.

SAD(|I _n(x)-I _n-1(x)|>MaxSAD).and.SAD(|I _n(x)-I _n-2(x)|>MaxSAD)

Wherein, I _n(x) be illustrated in t=n constantly, the position is the gray value of the pixel of x.

After step 103 is finished motion detection, can carry out attributive classification to each piece in the current frame image.Described block sort process can reach two purposes.

(1) as mentioned before, need seek flat site based on the Noise Estimation of flat site.Therefore, when the module based on the Noise Estimation of flat site was activated, the block sort module realized the searching task of flat block.According to the mean square deviation that the results of intermediate calculations of Wiener filtering is added up all pieces, the piece that possesses less relatively mean square deviation is considered to flat block.

(2) another purpose is the mode classification of certain piece can be offered follow-up piece reprocessing model selection and piece renewal model selection as parameter.Detailed process describes in detail in the back.

Step 104, the attribute that obtains at detection are static piece or pixel, carry out the renewal of background model; Described background model comprises a reference layer and at least one temporary layer, and described reference layer is used to store main reference data, and described temporary layer is used to store the auxiliary reference data; The renewal of described reference layer need be used corresponding auxiliary reference data in the described temporary layer;

Step 104 is used to finish the renewal process of background model.The proposition of the invention the adaptive background model of a kind of multilayer, simply be described below:

For each pixel, as shown in Figure 3, it has the n layer, a reference layer, n-1 temporary layer.Temporary layer is to be used for preserving the static and temporary transient time domain statistics that does not have reference significance in the short time that has reference significance in the time in the past section, and reference layer is to be used for being kept at the time domain statistics that in the current slot time-domain filtering is had reference significance.The number of layer is by the complexity decision of background, general layer 2-4.

For the real-time of implementation algorithm, the foundation of background model can block-based unit.

In a preferred embodiment of the invention, for each piece, can use five parameters to be described its current state.ModeBeFixed, whether the reference layer of describing current block is stable, promptly whether has reference significance.ModeStatistic is the statistical number that the state to current reference layer keeps always.BeResetOrUpdata is a update strategy of describing the reference layer of current block, resets again or time domain iteration renewal current block reference layer data.ICompensatedDUNCAN is to describe the current block motion match result that filtering is provided at DUNCAN.BeStillBackground is to describe current block whether to belong to static background.

In addition, for all pieces, there is a motion detection threshold value MaxSAD; For all pixels, there is a deviation range threshold T _Noise, both sizes all are directly proportional with noise intensity (being noise level parameter of the present invention).These two parameters all are the global constants that changes with noise intensity, rather than as constant in the background model of seeing in the past.This point, background model different in video denoising and video analysis are used just.

At the update strategy of the background model of video denoising, its purpose is to keep as much as possible the time domain statistics that follow-up filtering is had reference significance.For the video denoising, the object that is moving is not possess reference significance on time domain, so the first step of update background module is to seek the data of static region.With the piece is that unit classifies, and after obtaining static data, the pixel of static block is carried out model modification.The more new technological process of multilayer adaptive background model, as shown in Figure 4.

In a preferred embodiment of the present invention, can be according to the difference of block sort, from two kinds more the selection one of new model carry out.Pattern one, the time domain statistics of reference layer data is upgraded; Pattern two, the replacement of each layer data.For example, the attribute that obtains at detection is static piece or pixel, finish the renewal of background model by following steps:, then upgrade the related data in its affiliated background model if the data of current block or pixel have identical time domain statistical property with corresponding reference layer data; Otherwise, with in the existing background model effectively the corresponding data in the temporary layer of reference value minimum abandon, with the data of current block or pixel data as this temporary layer.

Preferably, finish after the renewal of the related data in the background model under current block or the pixel, also comprise: when the steadiness parameter of this piece or pixel conforms to a predetermined condition, then the related data with the temporary layer of this piece or pixel places reference layer.

Particularly,

For the piece of thinking " motion " after the motion detection, do not participate in the renewal of multilayer background model, its beResetOrUpdata of mark is False, after static appearance next time, current reference layer data are detected, confirm whether it possesses identical time domain with reference to characteristic with current reference layer data.

For the piece of thinking " static state " after the motion detection.If the beResetOrUpdata of current block is Ture, think that then current data and reference layer have identical time domain statistical property, then adopt pattern one to upgrade; Otherwise think that current data is uncertain with the identical time domain statistical property of reference layer, at first calculate the SAD between current data and the background data then, if its SAD is greater than MaxSAD, think that then the time domain specification of current block changes, then adopt pattern two to upgrade, otherwise employing pattern one is upgraded.

Respectively above-mentioned pattern one and pattern two are simply introduced below.

(1) time domain of reference layer data statistics is upgraded:

At first, with the gray value B of reference layer _n(x) compare, if | I _n(x)-B _n(x) |＜=T _NoiseSet up, then carry out following two data and handle:

(1)B _n+1(x)＝αB _n(x)+(1-α)I _n(x)

(2) current block modeStatistic=modeStatistic+1

Wherein, I _n(x) be illustrated in t=n constantly, the position is the gray value of the pixel of x; α is a time constant, the renewal speed of information that its expression is old and new information.

And then, also need to carry out following operation:

If current block belongs to stable piece, promptly modeBeFixed is Ture, if modeStatistic so〉max Fixed, then with all temporary layer data recursion one deck successively upwards.

If current block belongs to astable, if modeStatistic so〉max Initial, then the characterising parameter modeBeFixed of current reference layer is changed to Ture.

Whether belong to stable piece for current block, then can judge to get final product, for example, in the frame scope of inactive state in predetermined number of this piece, all do not change, determine that then this piece belongs to stable piece by pre-defined rule, otherwise, belong to astable.

(2) each layer data of resetting:

Gray value B with all layers (comprising temporary layer and reference layer) _n(x) relatively, if there is B _n(x) satisfy following expression:

|I _n(x)-B _n(x)|<＝T _noise

Then select the immediate B of current data _n(x), order

B ₀(x)＝B _n(x)，

Wherein, B ₀(x) be illustrated in t=0 constantly, the position is the time domain reference gray level value of the pixel of x.

If do not satisfy aforesaid expression formula: | I _n(x)-B _n(x) |＜=T _NoiseThen, order

B ₀(x)＝I _n(x)

According to above as can be known, data must just may be transformed into the reference layer data under the situation of continous-stable.And have the data of reference significance, and its level is not low more, and its protected degree is high more.The data-at-rest of each frame is introduced in the multilayer background model, has realized the renewal to the multilayer background model.This mechanism just guaranteed the reasonability of background model data, avoided noise in the static background to introduce and the introducing of brief fluctuations data.

If the attribute of step 105 object block or pixel is static, and the intensity of variation of its corresponding reference layer data conforms to a predetermined condition, and then the reference data that is provided according to the background model reference layer is carried out filtering; Otherwise, carry out common filtering.Step 105 is selected different filtering modes according to the related data of current block and background model, and is carried out; Wherein, the mode of described common filtering can comprise: any in linear low-pass filtering algorithm, non-linear sort algorithm or the 3D-DUNCAN filtering algorithm.

With reference to Fig. 5, show the reprocessing schematic flow sheet of the preferred embodiment of simple employing 3D-DUNCAN algorithm filtering.The relation that mainly is based between background model and the present frame shown in Figure 5 realizes the filtering of present frame.3D-DUNCAN filtering is based on the DUNCAN filtering of interframe movement coupling.Motion match makes that the homologue body region of window of DUNCAN filtering of interframe is approaching as far as possible, makes that promptly have more in the data with respect to the window of the DUNCAN filtering of other frames of present frame can be with reference to property.

In the embodiment of 3D-DUNCAN algorithm filtering, whether the reprocessing model selection of current block has the reference background meaning according to current block exactly, and the branch that finishes subsequent treatment is selected.If the background model of current data possesses reference significance and current block is static, then utilize current data block and current background reference data to be weighted average filter, finish the filtering of current block; Otherwise, in time domain, carry out the piece estimation, utilize data to carry out 3D-DUNCAN filtering with correlation.

The result of piece estimation is used for follow-up piece 3D-DUNCAN filtering.Consider that excessive motion vector can have influence on the precision of subsequent treatment, the piece estimation is only carried out among a small circle at one, for example-8 to+8.Diamond search (ds) is a kind of searching algorithm of motion among a small circle that is applicable to, therefore be can be used as an advantageous measure of the present invention.

In time-domain filtering based on the piece of background model, I _n(x) be illustrated in t=n constantly, the position is the gray values of pixel points of x.B _n(x) be illustrated in t=n constantly, the position is the time domain reference gray level value of the pixel of x, then

$I_n\left(x\right)=\left\{\begin{array}{ccccc}\mathrm{\&beta;}{I}_n\left(x\right)+(1-\mathrm{\&beta;})B_n\left(x\right)& \mathrm{if}& x& \mathrm{is}& \mathrm{non}-\mathrm{moving}\\ I_n\left(x\right)& \mathrm{if}& x& \mathrm{is}& \mathrm{moving}\end{array}\right.$

Wherein, β represents the weight of former gray value, and its size depends on the stability of background model.If the stability of background model is high, the β value is less so, even assignment is 0.

DUNCAN filtering is based on the raw image data of current window and the data of the filtered reference windows of former frame, select wherein valid data according to specific window mask, distribute according to certain weights then to be weighted on average, thereby obtain filtered result.1), may have base point in the window and exceeded reasonable range the setting of window mask is based on 2 points:; 2) may some point not to be in the same object in the window, with current pending point.Therefore, it is very necessary selecting rational data to participate in computing by the window mask.

The algorithm that this paper recommends is at zero-mean additive white Gaussian noise (AWGN),

$G\left(x\right)=\frac{1}{\mathrm{\&sigma;}\sqrt{2\mathrm{\&pi;}}}{e}^{-\frac{{(x-\mathrm{\&mu;})}^2}{2{\mathrm{\&sigma;}}^2}}$

According to its statistical distribution characteristic, following rule is effective:

1), 68% sample point is in [μ-σ, μ+σ] scope

2), 95% sample point is in [μ-2 σ, μ+2 σ] scope

3), 99.7% sample point is in [μ-3 σ, μ+3 σ] scope

Wherein, σ is exactly the aforementioned noise level parameter that obtains; μ is meant the intermediate value of noise energy.

Based on characteristic described above, DUNCAN filtering can be selected data in the following manner:

CurrPix represents current pending pixel.Selecting data, is to be the center with three values (CurrPix, CurrPix+ σ, CurrPix-σ), with S is to carry out in three zones of width, as shown in Figure 6.Width S is directly proportional with σ, and σ is big more, and S is big more.The value of S is a principle to obtain maximum pixels that rationally exists.By the Gaussian Profile characteristic as can be known, even current pending pixel is a noise spot, can guarantee that still most of points of choosing are near the Gaussian Profile center.In addition, even current pending point is in certain border, perhaps certain error has appearred in motion match, the DUNCAN filter still can reasonably select with current pending spot correlation have a few.

Crucial effect has also been played in being distributed in of weights in the DUNCAN filtering.Adjust with reference to the weights α fibrous root of the intraframe data minimum SAD size after according to motion match.If its SAD greater than predetermined threshold value, then reduces to be changed to 0 with reference to the weights of intraframe data or with it.Weights α not only depends on the size of SAD, also depends on the residing scope of pixel.Final filtering result is:

$\mathrm{FVal}=\frac{\underset{i}{\mathrm{\&Sigma;}}{\mathrm{\&alpha;}}_i{X}_i}{\underset{i}{\mathrm{\&Sigma;}}{\mathrm{\&alpha;}}_i}$

Wherein, X _iThe pixel that expression selects the back to keep through the window mask; α _iExpression and X _iCorresponding weights.

Need to prove that top embodiment has directly adopted the operational pattern at piece in description, but in fact the present invention also can adopt direct operational pattern at pixel fully.In a preferred embodiment of the invention, recommending the operational pattern of selection at piece, mainly is to consider the arithmetic speed problem, determines for better real-time property; In addition, it is suitable that the if block size is selected, and its performance is with approaching substantially based on the algorithm of pixel, and then can reduce much its operation time that takies.

In addition, in the preliminary treatment filtering and in the post processing and filtering process, also there are other a lot of modes in Wiener filtering and DUNCAN filtering except the present invention mentions, and the selection of filter is a kind of based on the performance of application scenarios and the compromise of computing time.

With reference to Fig. 7, show a kind of method embodiment that optimizes background model, specifically can may further comprise the steps:

Attribute is static piece or pixel in step 701, the present frame that obtains at motion detection, judges whether the data of current block or pixel have identical time domain statistical property with corresponding reference layer data; If then setting up procedure 702, otherwise, setting up procedure 703;

Step 702, statistics are upgraded the related data in its affiliated background model;

Step 703, with in the existing background model effectively the corresponding data in the temporary layer of reference value minimum abandon, with the data of current block or pixel data as this temporary layer.

Wherein, described background model comprises a reference layer and at least one temporary layer, and described reference layer is used to store main reference data, and described temporary layer is used to store the auxiliary reference data; The renewal of described reference layer need be used corresponding auxiliary reference data in the described temporary layer.

Preferably, after the renewal of the related data in the background model under finishing current block or pixel, can also comprise: when the steadiness parameter of this piece or pixel conforms to a predetermined condition, then the related data with the temporary layer of this piece or pixel places reference layer.

Object lesson and detailed description at background model is upgraded relate in front, do not repeat them here.

For aforesaid each method embodiment, for simple description, so it all is expressed as a series of combination of actions, but those skilled in the art should know, the present invention is not subjected to the restriction of described sequence of movement, because according to the present invention, some step can adopt other orders or carry out simultaneously.

With reference to Fig. 8, show a kind of system embodiment of video data real-time de-noising, comprising:

Noise Estimation module 801 is used for image data, carries out noise energy and estimates the output noise extent index;

Preliminary treatment filtration module 802 is used for finishing preliminary treatment filtering according to described noise level parameter at current frame image;

Piece motion detection block 803 is used for carrying out the piece motion detection at pretreated image; Can be understood as and seeking immediate reference data on the time domain;

Block sort module 804 is used for carrying out based on the result of motion detection the division of piece attribute; In fact, block sort module 804 can be understood as based on the result of motion detection carry out background area and non-background area division, have the division of time domain reference significance zone and non-time domain reference zone;

Background model update module 805, the attribute that is used for obtaining at detection is static piece or pixel, carries out the renewal of background model; Described background model comprises a reference layer and at least one temporary layer, and described reference layer is used to store main reference data, and described temporary layer is used to store the auxiliary reference data; The renewal of described reference layer need be used corresponding auxiliary reference data in the described temporary layer;

Post processing and filtering module 806, the attribute that is used for when object block or pixel is static, and the intensity of variation of its corresponding reference layer data carries out filtering according to the reference data that the background model reference layer is provided when conforming to a predetermined condition; Otherwise, carry out common filtering.Wherein, the mode of described common filtering comprises: any in linear low-pass filtering algorithm, non-linear sort algorithm or the 3D-DUNCAN filtering algorithm.

Further, post processing and filtering module 806 can comprise two submodules: first filter 8061 is used for carrying out time-space domain filtering according to the reference data that the background model reference layer is provided; Second filter 8062, but be used to adopt of the denoising of common filtering algorithm to other All Rangeses outside the background area with reference significance.

Preferably, the attribute that obtains at detection is static piece or pixel, and described background model update module 805 can comprise:

Judge submodule 8051, be used to judge whether the data of current block or pixel have identical time domain statistical property with corresponding reference layer data; If, then start first updating submodule, otherwise, second updating submodule started;

First updating submodule 8052 is used for adding up the related data of upgrading its affiliated background model;

Second updating submodule 8053, be used for existing background model effectively the corresponding data in the temporary layer of reference value minimum abandon, with the data of current block or pixel data as this temporary layer.

Further, described background model update module can also comprise: data advance submodule 8054, be used for after the renewal of related data of the background model under finishing current block or pixel, when the steadiness parameter of this piece or pixel conformed to a predetermined condition, then the related data with the temporary layer of this piece or pixel placed reference layer.

Need to prove that those skilled in the art should know, the embodiment described in the specification all belongs to preferred embodiment, and related action and module might not be that the present invention is necessary.

Secondly, each embodiment in this specification all adopts the mode of going forward one by one to describe, and what each embodiment stressed all is and the difference of other embodiment that identical similar part is mutually referring to getting final product between each embodiment.For system embodiment, because it is similar substantially to method embodiment, so description is fairly simple, relevant part gets final product referring to the part explanation of method embodiment.

In order to estimate performance of the present invention, this paper has adopted several standard test sequences, and to these sequences average is 0 in addition, intensity is respectively Gauss's additive white noise of 10,15,20, directly use 3D-DUNCAN to carry out denoising then respectively and the present invention's (a plurality of steps that comprise 3D-DUNCAN) carries out denoising.

In simulation process, the pixel number of plies is set at 3 layers, and the max Initial of background model initializing process is 15 frames; Max Fixed after background model is stable is set at 20 frames; In the background model renewal process, the time, update coefficients α was set at 1/8; Only adopt adjacent continuous three frame data when in last handling process, relating to motion match, reduced the data volume that needs processing as far as possible.

Following table shows the present invention and directly adopts the performance of 3D-DUNCAN algorithm to compare, and its mean P SNR numerical value is based on the above data statistics value of 50 frames.The present invention adopts AMBVD to represent (Adaptive Multi-layer Background Video De-noising is based on the real-time video Denoising Algorithm of multilayer adaptive background model) in table.Wherein, the contaminated degree of meansquaredeviation presentation video of the noise in the table.

In a word, the real-time video Denoising Algorithm based on multilayer adaptive background model disclosed in this invention is that strong, the subjective objective effect of a kind of applicability is significant, has the video Denoising Algorithm of real-time, especially at there being the video sequence of stablizing background.Multilayer adaptive background model is the background model of setting up at the video Denoising Algorithm.This model has played significant inhibition effect to the noise between frames of video sequence.The update mechanism of multilayer adaptive background model has been guaranteed still can set up stable background model after scene is switched, and can satisfy the lower application scenarios of scene change frequency, thereby strengthen the robustness of video Denoising Algorithm to scene.Aforesaid experimental result shows that more also technical scheme of the present invention can provide the high-quality image quality.

More than to the method and system of a kind of video data real-time de-noising provided by the present invention, and a kind of method of optimizing background model, be described in detail, used specific case herein principle of the present invention and execution mode are set forth, the explanation of above embodiment just is used for helping to understand method of the present invention and core concept thereof; Simultaneously, for one of ordinary skill in the art, according to thought of the present invention, the part that all can change in specific embodiments and applications, in sum, this description should not be construed as limitation of the present invention.

Claims (10)

1, a kind of method of video data real-time de-noising is characterized in that, comprising:

Image data is carried out noise energy and is estimated the output noise extent index;

According to described noise level parameter, finish preliminary treatment filtering at current frame image;

At pretreated image, carry out motion detection;

The result of described motion detection is carried out the attributive classification of each piece, and be static piece or pixel, carry out the renewal of background model at attribute; Described background model comprises a reference layer and at least one temporary layer, and described reference layer is used to store main reference data, and described temporary layer is used to store the auxiliary reference data; The renewal of described reference layer need be used corresponding auxiliary reference data in the described temporary layer;

If the attribute of object block or pixel is static, and the intensity of variation of its corresponding reference layer data conforms to a predetermined condition, and then the reference data that is provided according to the background model reference layer is carried out filtering; Otherwise, carry out common filtering.

2, the method for claim 1 is characterized in that, described is static piece or pixel at attribute, finishes the renewal of background model by following steps:

If the data of current block or pixel have identical time domain statistical property with corresponding reference layer data, then upgrade the related data in its affiliated background model;

Otherwise, with in the existing background model effectively the corresponding data in the temporary layer of reference value minimum abandon, with the data of current block or pixel data as this temporary layer.

3, method as claimed in claim 2 is characterized in that, finishes after the renewal of the related data in the affiliated background model of current block or pixel, also comprises:

When the steadiness parameter of this piece or pixel conforms to a predetermined condition, then the related data with the temporary layer of this piece or pixel places reference layer.

4, the method for claim 1 is characterized in that, the mode of described common filtering comprises:

In linear low-pass filtering algorithm, non-linear sort algorithm or the 3D-DUNCAN filtering algorithm any.

5, a kind of system of video data real-time de-noising is characterized in that, comprising:

The Noise Estimation module is used for image data, carries out noise energy and estimates the output noise extent index;

The preliminary treatment filtration module is used for finishing preliminary treatment filtering according to described noise level parameter at current frame image;

The piece motion detection block is used for carrying out the piece motion detection at pretreated image;

The block sort module is used for the result of described motion detection is carried out the attributive classification of each piece;

The background model update module, being used at attribute is static piece or pixel, carries out the renewal of background model; Described background model comprises a reference layer and at least one temporary layer, and described reference layer is used to store main reference data, and described temporary layer is used to store the auxiliary reference data; The renewal of described reference layer need be used corresponding auxiliary reference data in the described temporary layer;

The post processing and filtering module, the attribute that is used for when object block or pixel is static, and the intensity of variation of its corresponding reference layer data carries out filtering according to the reference data that the background model reference layer is provided when conforming to a predetermined condition; Otherwise, carry out common filtering.

6, system as claimed in claim 5 is characterized in that, described background model update module comprises:

Judge submodule, be used to judge whether the data of current block or pixel have identical time domain statistical property with corresponding reference layer data; If, then start first updating submodule, otherwise, second updating submodule started;

First updating submodule is used for adding up the related data of upgrading its affiliated background model;

Second updating submodule, be used for existing background model effectively the corresponding data in the temporary layer of reference value minimum abandon, with the data of current block or pixel data as this temporary layer.

7, system as claimed in claim 6 is characterized in that, described background model update module also comprises:

Data advance submodule, be used for after the renewal of related data of the background model under finishing current block or pixel, when the steadiness parameter of this piece or pixel conformed to a predetermined condition, then the related data with the temporary layer of this piece or pixel placed reference layer.

8, system as claimed in claim 5 is characterized in that, the mode of described common filtering comprises:

In linear low-pass filtering algorithm, non-linear sort algorithm or the 3D-DUNCAN filtering algorithm any.

9, a kind of method of optimizing background model is characterized in that, described background model comprises a reference layer and at least one temporary layer, and described reference layer is used to store main reference data, and described temporary layer is used to store the auxiliary reference data; The renewal of described reference layer need be used corresponding auxiliary reference data in the described temporary layer; Described method comprises:

Carry out the attributive classification of each piece at the result of motion detection, and be static piece or pixel, judge whether the data of current block or pixel have identical time domain statistical property with corresponding reference layer data at attribute in the present frame;

If then statistics is upgraded the related data in its affiliated background model;

Otherwise, with in the existing background model effectively the corresponding data in the temporary layer of reference value minimum abandon, with the data of current block or pixel data as this temporary layer.

10, method as claimed in claim 9 is characterized in that, finishes after the renewal of the related data in the affiliated background model of current block or pixel, also comprises:

When the steadiness parameter of this piece or pixel conforms to a predetermined condition, then the related data with the temporary layer of this piece or pixel places reference layer.

Images