CN102685370B - De-noising method and device of video sequence - Google Patents

Abstract

The invention discloses a de-noising method and a de-noising device of a video sequence. The de-noising method comprises the following steps of: firstly, decomposing the video sequence into a low-rank part and a sparse part; decomposing the sparse part into a video self-characteristic part and a noise part; and determining the de-noised video sequence according to the low-rank part and the video self-characteristic part. The technical scheme provided by the embodiment can effectively remove noises in the video sequence; and a process of removing the noises in the video sequence does not need to accurately estimate the noise strength, so that the problem that the noise removing effect is not ideal, caused by estimation errors of the noise strength, is solved.

Description

A kind of denoising method of video sequence and device

Technical field

The present invention relates to a kind of technical field of video processing, relate in particular to a kind of denoising method and device of video sequence.

Background technology

Because video acquisition transmission course ambient light is bad, perhaps can there be corresponding noise in the problem of imaging device itself in the common video sequence that gathers.Noise in the video sequence can cause video quality to descend on the one hand, and the difficulty that also may aggravate on the other hand Video coding is so that the Video coding Efficiency Decreasing.Therefore, need to go accordingly (falling) processing of making an uproar for corresponding video sequence (digital image sequence).

The target of carrying out denoising for video sequence is from being isolated as far as possible original video sequence the video sequence of noise pollution.Normally adopt at present the mode of matrix decomposition to remove noise in the video sequence.

Suppose to treat the total N frame of video sequence of denoising, wherein, the pixels tall of each video pictures is H, and width is W.Each video pictures is pulled into column vector, and note mi represents the column vector that i video pictures pulls into, and its dimension is that HW * 1(HW is capable, row).Further, N column vector m1～mN is combined into two-dimensional matrix M, and its dimension is that HW * N(HW is capable, the N row).

In the process of removing noise, directly M is decomposed into low-rank part L, sparse part S and Gaussian noise part Z, particularly, being based on RPCA(Robust Principle Component Analysis, the robust principal component analysis) decomposition method removes noise processed accordingly, namely find the solution L, S and Z according to following constraints, with the removal noise, and with noise Z constraint within the specific limits, this constraints is:

min||L|| _*+λ||S|| ₁

s.t.L+S+Z=M

||Z|| _F≤δ；

Wherein, || || _*The kernel of matrix norm, || || ₁The L1 norm of matrix, || || _FBe Fu Luobin Nice norm of matrix, λ is weight.δ is the noise intensity of estimating.Minimize accordingly nuclear norm || L|| _*Can guarantee the low-rank of L, minimize the L1 norm || S|| ₁Can guarantee the sparse property of S, || Z|| _F≤ δ so that the energy constraint of noise section Z in certain scope.

Can find out by foregoing description, when noise energy is smaller, use || Z|| _F≤ δ retrains the energy of noise section Z can obtain certain effect.Yet, when noise energy is larger, namely with signal energy relatively near the time, above-mentioned constraints will be so that the result of each expression formula and optimal result form larger deviation.Be to comprise L and S partial data among the noise Z.And, in the processing scheme of above-mentioned removal noise, also exist noise intensity and estimate that δ is not easy exactly determined problem.

Summary of the invention

The denoising method and the device that the purpose of this invention is to provide a kind of video sequence, thus can remove easily and effectively noise in the video sequence.

The objective of the invention is to be achieved through the following technical solutions:

A kind of denoising method of video sequence comprises:

Video sequence is decomposed into low-rank part and sparse part, is video unique characteristics part and noise section again with described sparse decomposed;

According to described low-rank part and the described video unique characteristics video sequence after definite denoising partly.

Alternatively, describedly video sequence is decomposed into low-rank part and sparse part comprises:

According to constraints: $\begin{array}{c}\min {\left|\right|L\left|\right|}_{*}+\mathrm{\λ}{\left|\right|S\left|\right|}_1\\ s.t.L+S=M\end{array}$ Determine L and S, wherein, L is that low-rank part, S are sparse part, and M is video sequence, and λ is weight.

Alternatively, the step of described definite L and S comprises:

Judge $\frac{{\left|\right|M-L-S\left|\right|}_F}{{\left|\right|M\left|\right|}_F}\≥{\mathrm{tol}}_1$ Whether set up:

If set up, then calculate L and S according to following formula:

$L=\underset{L}{\arg \min }({\left|\right|L\left|\right|}_{*}+\frac{c}{2}{\left|\right|L+\frac{P}{c}+S-M\left|\right|}_F^2);$

$S=\underset{S}{\arg \min }(\mathrm{\λ}{\left|\right|S\left|\right|}_1+\frac{c}{2}{\left|\right|S+\frac{P}{c}+L-M\left|\right|}_F^2);$

Also upgrade P:P=P+c (L+S-M), wherein, L=0 when initial, S=0, LaGrange parameter P=0, tol ₁=10 ^-2, c=1;

If be false, the L that then current calculating is obtained and S are as final calculation result.

Alternatively, described is that video unique characteristics part and noise section comprise with described sparse decomposed:

According to constraints: $\begin{array}{c}{\min \left|\right|S1\left(i\right)\left|\right|}_{\mathrm{TV}}+\mathrm{\μ}{\left|\right|S2\left(i\right)\left|\right|}_1\\ s.t.S1\left(i\right)+S2\left(i\right)=S\left(i\right)\end{array}$ Determine video unique characteristics part S1 (i) and the noise section S2 (i) of the sparse part correspondence of video sequence i frame, wherein, S (i) is described sparse part, and μ is weight;

After each self-corresponding video unique characteristics part of the sparse part of all frames in the video sequence and noise section all are determined, the decomposition result of the sparse part of all frames in the acquisition video sequence.

Alternatively, the step of described definite S1 (i) and S2 (i) comprises:

Judge $\frac{{\left|\right|S\left(i\right)-S1\left(i\right)-S2\left(i\right)\left|\right|}_F}{{\left|\right|S\left(i\right)\left|\right|}_F}\≥{\mathrm{tol}}_2$ Whether set up:

If set up, then calculate S1 (i) and S2 (i) according to following formula:

$S1\left(i\right)=\underset{S1\left(i\right)}{\arg \min }({\left|\right|S1\left(i\right)\left|\right|}_{\mathrm{TV}}+\frac{d}{2}{\left|\right|S1\left(i\right)+\frac{Q}{d}+S2\left(i\right)-S\left(i\right)\left|\right|}_F^2);$

$S2\left(i\right)=\underset{S2\left(i\right)}{\arg \min }({\mathrm{\μ}\left|\right|S2\left(i\right)\left|\right|}_1+\frac{d}{2}{\left|\right|S2\left(i\right)+\frac{Q}{d}+S1\left(i\right)-S\left(i\right)\left|\right|}_F^2);$

Also upgrade Q:Q=Q+d (S1 (i)+S2 (i)-S (i)), wherein, S1 (i)=0 when initial, S2 (i)=0, LaGrange parameter Q=0, tol ₂=10 ^-7, d=1;

If be false, the S1 (i) that then current calculating is obtained and S2 (i) are as final calculation result.

A kind of denoising device of video sequence comprises:

Video processing module is used for video sequence is decomposed into low-rank part and sparse part;

Sparse section processes module, being used for the sparse decomposed that described video processing module obtains is video unique characteristics part and noise section;

The denoising module is used for the partly video sequence after definite denoising of video unique characteristics that the low-rank part that obtains according to described video processing module and described sparse section processes module obtain.

Alternatively, described video processing module specifically is used for according to constraints: $\begin{array}{c}\min {\left|\right|L\left|\right|}_{*}+\mathrm{\λ}{\left|\right|S\left|\right|}_1\\ s.t.L+S=M\end{array}$ Determine L and S, wherein, L is that low-rank part, S are sparse part, and M is video sequence, and λ is weight.

Alternatively, the step of described definite L and S comprises:

Judge $\frac{{\left|\right|M-L-S\left|\right|}_F}{{\left|\right|M\left|\right|}_F}\≥{\mathrm{tol}}_1$ Whether set up:

If set up, then calculate L and S according to following formula:

$L=\underset{L}{\arg \min }({\left|\right|L\left|\right|}_{*}+\frac{c}{2}{\left|\right|L+\frac{P}{c}+S-M\left|\right|}_F^2);$

$S=\underset{S}{\arg \min }(\mathrm{\λ}{\left|\right|S\left|\right|}_1+\frac{c}{2}{\left|\right|S+\frac{P}{c}+L-M\left|\right|}_F^2);$

Also upgrade P:P=P+c (L+S-M), wherein, L=0 when initial, S=0, LaGrange parameter P=0, tol ₁=10 ^-2, c=1:

If be false, the L that then current calculating is obtained and S are as final calculation result.

Alternatively, described sparse section processes module specifically is used for according to constraints: $\begin{array}{c}{\min \left|\right|S1\left(i\right)\left|\right|}_{\mathrm{TV}}+\mathrm{\μ}{\left|\right|S2\left(i\right)\left|\right|}_1\\ s.t.S1\left(i\right)+S2\left(i\right)=S\left(i\right)\end{array}$ Determine video unique characteristics part S1 (i) and the noise section S2 (i) of the sparse part correspondence of video sequence i frame, wherein, S (i) is described sparse part, and μ is weight; After each self-corresponding video unique characteristics part of the sparse part of all frames in the video sequence and noise section all are determined, the decomposition result of the sparse part of all frames in the acquisition video sequence.

Alternatively, the step of described definite S1 (i) and S2 (i) comprises:

Judge $\frac{{\left|\right|S\left(i\right)-S1\left(i\right)-S2\left(i\right)\left|\right|}_F}{{\left|\right|S\left(i\right)\left|\right|}_F}\≥{\mathrm{tol}}_2$ Whether set up:

If set up, then calculate S1 (i) and S2 (i) according to following formula:

$S1\left(i\right)=\underset{S1\left(i\right)}{\arg \min }({\left|\right|S1\left(i\right)\left|\right|}_{\mathrm{TV}}+\frac{d}{2}{\left|\right|S1\left(i\right)+\frac{Q}{d}+S2\left(i\right)-S\left(i\right)\left|\right|}_F^2);$

$S2\left(i\right)=\underset{S2\left(i\right)}{\arg \min }({\mathrm{\μ}\left|\right|S2\left(i\right)\left|\right|}_1+\frac{d}{2}{\left|\right|S2\left(i\right)+\frac{Q}{d}+S1\left(i\right)-S\left(i\right)\left|\right|}_F^2);$

Also upgrade Q:Q=Q+d (S1 (i)+S2 (i)-S (i)), wherein, S1 (i)=0 when initial, S2 (i)=0, LaGrange parameter Q=0, tol ₂=10 ^-7, d=1;

If be false, the S1 (i) that then current calculating is obtained and S2 (i) are as final calculation result.

As seen from the above technical solution provided by the invention, the technical scheme that the embodiment of the invention provides can effectively be removed the noise in the video sequence, and, in removing video sequence in the process of noise, do not need accurate estimating noise intensity, thereby avoid occurring the undesirable problem of noise remove effect that causes because of the noise intensity evaluated error, namely overcome the corresponding problem that exists in the prior art.

Description of drawings

In order to be illustrated more clearly in the technical scheme of the embodiment of the invention, the accompanying drawing of required use was done to introduce simply during the below will describe embodiment, apparently, accompanying drawing in the following describes only is some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite of not paying creative work, can also obtain other accompanying drawings according to these accompanying drawings.

The processing procedure schematic diagram of the method that Fig. 1 provides for the embodiment of the invention;

The structural representation of the device that Fig. 2 provides for the embodiment of the invention;

Fig. 3 is the Application Example schematic diagram of the embodiment of the invention;

Fig. 4 is the application scenarios schematic diagram one of the embodiment of the invention;

Fig. 5 is the application scenarios schematic diagram two of the embodiment of the invention;

Fig. 6 is the application scenarios schematic diagram three of the embodiment of the invention;

Fig. 7 is the application scenarios schematic diagram four of the embodiment of the invention.

Embodiment

Below in conjunction with the accompanying drawing in the embodiment of the invention, the technical scheme in the embodiment of the invention is clearly and completely described, obviously, described embodiment only is the present invention's part embodiment, rather than whole embodiment.Based on embodiments of the invention, those of ordinary skills belong to protection scope of the present invention not making the every other embodiment that obtains under the creative work prerequisite.

The embodiment of the invention has specifically been introduced the mode of finding the solution step by step and has been obtained noise in the video sequence, thereby is convenient to effectively remove the noise in the video sequence.Below in conjunction with accompanying drawing the embodiment of the invention is described in further detail.

As shown in Figure 1, the denoising method of a kind of video sequence of providing of the embodiment of the invention specifically comprises:

Step 11 is decomposed into low-rank part and sparse part with video sequence;

Be that video sequence only is decomposed into low-rank part and sparse part in this step, do not decomposite noise section, accordingly video sequence is decomposed into low-rank partly and the mode of sparse part specifically can but be not limited in the following ways:

According to constraints: $\begin{array}{c}\min {\left|\right|L\left|\right|}_{*}+\mathrm{\λ}{\left|\right|S\left|\right|}_1\\ s.t.L+S=M\end{array}$ Determine L and S, wherein, L is that low-rank part, S are sparse part, and M is video sequence, and λ is weight, and this weighted value can rule of thumb be determined.

The process of determining L and S based on above-mentioned constraints specifically can but be not limited to comprise:

Judge

Whether set up, wherein, the tol in this inequality ₁=10 ^-2, perhaps this parameter also can be other predetermined values, and carries out following operation according to judged result, that is:

Set up if judge above-mentioned inequality, then calculate L and S according to following formula:

$L=\underset{L}{\arg \min }({\left|\right|L\left|\right|}_{*}+\frac{c}{2}{\left|\right|L+\frac{P}{c}+S-M\left|\right|}_F^2);$

$S=\underset{S}{\arg \min }(\mathrm{\λ}{\left|\right|S\left|\right|}_1+\frac{c}{2}{\left|\right|S+\frac{P}{c}+L-M\left|\right|}_F^2);$

At this moment, also need to upgrade P:P=P+c (L+S-M), wherein, when initial: L=0, S=0, LaGrange parameter P=0, L, the S that adopts in the corresponding formula right side in the computational process afterwards and P are the last value that obtains, the c=1 in the above-mentioned formula of calculating;

Be false if judge above-mentioned inequality, the L that then current calculating is obtained and S be as final calculation result, namely until above-mentioned inequality when being false, be L and the S that decomposes acquisition for video sequence just calculate the L and the S that obtain.

Step 12 is video unique characteristics part and noise section with above-mentioned sparse decomposed;

In this step, be that the noise with sparse part decomposes out so that remove corresponding noise, with sparse decomposed be accordingly video unique characteristics part and noise section mode specifically can but be not limited in the following ways:

According to constraints: $\begin{array}{c}{\min \left|\right|S1\left(i\right)\left|\right|}_{\mathrm{TV}}+\mathrm{\μ}{\left|\right|S2\left(i\right)\left|\right|}_1\\ s.t.S1\left(i\right)+S2\left(i\right)=S\left(i\right)\end{array}$ Determine video unique characteristics part S1 (i) and the noise section S2 (i) of the sparse part correspondence of the i frame in the video sequence, wherein, S (i) is described sparse part, and μ is weight, and this weighted value can rule of thumb be determined; After each self-corresponding video unique characteristics part of the sparse part of all frames in the video sequence and noise section all are determined, the decomposition result of the sparse part of all frames in the acquisition video sequence.

The process of determining S1 (i) and S2 (i) based on above-mentioned constraints specifically can but be not limited to comprise:

Judge Whether set up, wherein, the tol in this inequality ₂=10 ^-7, perhaps this parameter also can be other predetermined values, and carries out following operation according to judged result, that is:

Set up if judge above-mentioned inequality, then calculate S1 (i) and S2 (i) according to following formula:

$S1\left(i\right)=\underset{S1\left(i\right)}{\arg \min }({\left|\right|S1\left(i\right)\left|\right|}_{\mathrm{TV}}+\frac{d}{2}{\left|\right|S1\left(i\right)+\frac{Q}{d}+S2\left(i\right)-S\left(i\right)\left|\right|}_F^2);$

$S2\left(i\right)=\underset{S2\left(i\right)}{\arg \min }({\mathrm{\μ}\left|\right|S2\left(i\right)\left|\right|}_1+\frac{d}{2}{\left|\right|S2\left(i\right)+\frac{Q}{d}+S1\left(i\right)-S\left(i\right)\left|\right|}_F^2);$

Simultaneously, also need to upgrade Q:Q=Q+d (S1 (i)+S2 (i)-S (i)), wherein, when initial: S1 (i)=0, S2 (i)=0, LaGrange parameter Q=0, S1 (i), the S2 (i) that adopts in the corresponding formula right side in the computational process afterwards and Q are the last value that obtains, the d=1 in the above-mentioned formula of calculating;

Be false if judge above-mentioned inequality, the S1 (i) that then current calculating is obtained and S2 (i) are as final calculation result.

Step 13, the video unique characteristics that the low-rank part that obtains according to step 11 and step 12 obtain is the video sequence after definite denoising partly, is about to the i frame that corresponding low-rank part L (i) and video unique characteristics part S1 (i) addition just can obtain to remove the video sequence behind the noise.

Carry out the Transformatin of the noise in the video sequence by above-mentioned implementation, can overcome problems of the prior art, thereby effectively improve the effect of in video sequence, removing noise.

The embodiment of the invention also provides a kind of denoising device of video sequence, and its implementation structure specifically can comprise following processing module as shown in Figure 2:

(1) video processing module 21, are used for video sequence is decomposed into low-rank part and sparse part;

This video processing module 21 specifically can but be not limited to for according to constraints: $\begin{array}{c}\min {\left|\right|L\left|\right|}_{*}+\mathrm{\λ}{\left|\right|S\left|\right|}_1\\ s.t.L+S=M\end{array}$ Determine L and S, wherein, L is that low-rank part, S are sparse part, and M is video sequence, and λ is weight.And in this video processing module 21, determine accordingly L and S operation specific implementation can but be not limited to comprise:

The process of determining L and S based on above-mentioned constraints specifically can but be not limited to comprise:

Judge

Whether set up, wherein, the tol in this inequality ₁=10 ^-2, perhaps this parameter also can be other predetermined values, and carries out following operation according to judged result, that is:

Set up if judge above-mentioned inequality, then calculate L and S according to following formula:

$L=\underset{L}{\arg \min }({\left|\right|L\left|\right|}_{*}+\frac{c}{2}{\left|\right|L+\frac{P}{c}+S-M\left|\right|}_F^2);$

$S=\underset{S}{\arg \min }(\mathrm{\λ}{\left|\right|S\left|\right|}_1+\frac{c}{2}{\left|\right|S+\frac{P}{c}+L-M\left|\right|}_F^2);$

At this moment, also need to upgrade P:P=P+c (L+S-M), wherein, when initial: L=0, S=0, LaGrange parameter P=0, L, the S that adopts in the corresponding formula right side in the computational process afterwards and P are the last value that obtains, the c=1 in the above-mentioned formula of calculating;

Be false if judge above-mentioned inequality, the L that then current calculating is obtained and S be as final calculation result, namely until above-mentioned inequality when being false, be L and the S that decomposes acquisition for video sequence just calculate the L and the S that obtain.

(2) sparse section processes module 22, being used for the sparse decomposed that above-mentioned video processing module 21 obtains is video unique characteristics part and noise section;

This sparse section processes module 22 specifically can but be not limited to for according to constraints: $\begin{array}{c}{\min \left|\right|S1\left(i\right)\left|\right|}_{\mathrm{TV}}+\mathrm{\μ}{\left|\right|S2\left(i\right)\left|\right|}_1\\ s.t.S1\left(i\right)+S2\left(i\right)=S\left(i\right)\end{array}$ Determine video unique characteristics part S1 (i) and the noise section S2 (i) of the sparse part correspondence of the i frame in the video sequence, wherein, S (i) is described sparse part, and μ is weight; After each self-corresponding video unique characteristics part of the sparse part of all frames in the video sequence and noise section all are determined, the decomposition result of the sparse part of all frames in the acquisition video sequence.And in this sparse section processes module 22, the mode of determining accordingly S1 (i) and S2 (i) specifically can but be not limited to comprise:

Judge

Whether set up, wherein, the tol in this inequality ₂=10 ^-7, perhaps this parameter also can be other predetermined values, and carries out following operation according to judged result, that is:

Set up if judge above-mentioned inequality, then calculate S1 (i) and S2 (i) according to following formula:

$S1\left(i\right)=\underset{S1\left(i\right)}{\arg \min }({\left|\right|S1\left(i\right)\left|\right|}_{\mathrm{TV}}+\frac{d}{2}{\left|\right|S1\left(i\right)+\frac{Q}{d}+S2\left(i\right)-S\left(i\right)\left|\right|}_F^2);$

$S2\left(i\right)=\underset{S2\left(i\right)}{\arg \min }({\mathrm{\μ}\left|\right|S2\left(i\right)\left|\right|}_1+\frac{d}{2}{\left|\right|S2\left(i\right)+\frac{Q}{d}+S1\left(i\right)-S\left(i\right)\left|\right|}_F^2);$

Simultaneously, also need to upgrade Q:Q=Q+d (S1 (i)+S2 (i)-S (i)), wherein, when initial: S1 (i)=0, S2 (i)=0, LaGrange parameter Q=0, S1 (i), the S2 (i) that adopts in the corresponding formula right side in the computational process afterwards and Q are the last value that obtains, the d=1 in the above-mentioned formula of calculating;

Be false if judge above-mentioned inequality, the S1 (i) that then current calculating is obtained and S2 (i) are as final calculation result.

(3) denoising module 23 is used for the partly video sequence after definite denoising of video unique characteristics that the low-rank part that obtains according to above-mentioned video processing module 21 and above-mentioned sparse section processes module 22 obtain; Particularly, corresponding low-rank part L (i) and video unique characteristics part S1 (i) addition just can be obtained remove the i frame of the video sequence behind the noise.

By the realization of said apparatus, can effectively remove the noise in the vision signal, and in removing the process of noise, not need the estimation of accurate noise intensity, thereby avoid occurring the undesirable problem of noise remove effect that causes because of the noise intensity evaluated error.

The embodiment of the invention at first, can use undemanding RPCA decomposition method that video sequence M is divided into low-rank part L and sparse part S in the specific implementation process; Afterwards, can carry out micronization processes to sparse part S by accurate TV-L1 decomposition method, the feature S1(that S is resolved into video itself is video unique characteristics part) and impulsive noise S2(be noise section); At last, just can be according to the vision signal behind L and the definite removal of the S1 noise.

Be described in detail below in conjunction with the specific implementation process of accompanying drawing to the embodiment of the invention.

Suppose to treat the total N frame of video sequence (take gray level image as example, coloured image can be regarded three gray level images of RGB as) of denoising, wherein, the pixels tall of each video pictures is H, and width is W.Each video pictures is pulled into column vector, and note mi represents the column vector that i video pictures pulls into, and its dimension is that HW * 1(HW is capable, row).Further, N column vector m1～mN is combined into two-dimensional matrix M, and its dimension is that HW * N(HW is capable, the N row).

With reference to shown in Figure 3, the specific implementation process of the embodiment of the invention can comprise:

(1) undemanding RPCA operation splitting process

For matrix M, can use following optimization constraints that it is decomposed into low-rank matrix L (being the low-rank part of video sequence), dimension is HW * N, and sparse part S(is the sparse part of video sequence), dimension is HW * N;

Corresponding constraints can for: $\begin{array}{c}\min {\left|\right|L\left|\right|}_{*}+\mathrm{\λ}{\left|\right|S\left|\right|}_1\\ s.t.L+S=M\end{array},$ Wherein, || || _*The kernel of matrix norm, || || ₁Be the L1 norm of matrix, λ is weight, and λ can but be not limited to use

Accordingly, specifically find the solution L and the S process can may further comprise the steps:

(11) initialization procedure

The parameter that needs in this process need initialization subsequent processes comprises initial L=0, initial S=0, initial LaGrange parameter P=0.Also obtain predefined two parameter value: tol ₁=10 ^-2And c=1;

(12) the calculation of parameter inequality that obtains according to initialization procedure

Whether set up, if set up, execution in step (13) then, otherwise, execution in step (14);

(13) if

Set up, then carry out following parameter updating operation:

Upgrade the L value, update mode can but be not limited to: $L=\underset{L}{\arg \min }({\left|\right|L\left|\right|}_{*}+\frac{c}{2}{\left|\right|L+\frac{P}{c}+S-M\left|\right|}_F^2),$ Wherein, || || _FIt is Fu Luobin Nice norm of matrix;

Upgrade the S value, update mode can but be not limited to: $S=\underset{S}{\arg \min }(\mathrm{\λ}{\left|\right|S\left|\right|}_1+\frac{c}{2}{\left|\right|S+\frac{P}{c}+L-M\left|\right|}_F^2);$

Upgrade the P value, update mode can but be not limited to: P=P+c (L+S-M);

After upgrading corresponding parameter L, S and P, re-execute step (12);

(14) with current L and the output of S value, as the decomposition result for matrix M, namely as the decomposition result for video sequence, comprise low-rank matrix L and sparse part S.

(2) accurate TV-L1 operation splitting process

For the sparse part S that said process (1) is tried to achieve, be that dimension is the two-dimensional matrix of HW * N, its each row S (i) expression be sparse part in the video pictures (i.e. a frame), dimension is HW * 1.For each S (i), i=1, N, use respectively accurate TV-L1 constraints that it is decomposed into feature S1 (i) (being video unique characteristics part) and the impulsive noise S2 (i) (being noise section) of video itself in this operating process, the sparse part that is about to all frames in the video sequence need to be decomposed into respectively each self-corresponding video unique characteristics part and noise section.

Corresponding constraints can for: $\begin{array}{c}{\min \left|\right|S1\left(i\right)\left|\right|}_{\mathrm{TV}}+\mathrm{\μ}{\left|\right|S2\left(i\right)\left|\right|}_1\\ s.t.S1\left(i\right)+S2\left(i\right)=S\left(i\right)\end{array},$ Wherein, || || _TVBe the TV norm of matrix, μ is weight, can but be not limited to use μ=2p+0.8, the impulsive noise density of this p value for estimating can be estimated obtain by known methods such as ACWMF;

Accordingly, specifically find the solution S1 (i) and S2 (i) process can may further comprise the steps:

(21) initialization procedure

The parameter that needs in this process need initialization subsequent processes comprises initial S1 (i)=0, initial S2 (i)=0, initial LaGrange parameter Q=0.Also obtain predefined two parameter value: tol ₂=10 ^-7And d=1;

(22) the calculation of parameter inequality that obtains according to initialization procedure

Whether set up, if set up, execution in step (23) then, otherwise, execution in step (24);

(23) if

Set up, then carry out following parameter updating operation:

Upgrade S1 (i) value, update mode can but be not limited to:

$S1\left(i\right)=\underset{S1\left(i\right)}{\arg \min }({\left|\right|S1\left(i\right)\left|\right|}_{\mathrm{TV}}+\frac{d}{2}{\left|\right|S1\left(i\right)+\frac{Q}{d}+S2\left(i\right)-S\left(i\right)\left|\right|}_F^2),$ Wherein, || || _FIt is Fu Luobin Nice norm of matrix;

Upgrade S2 (i) value, update mode can but be not limited to:

$S2\left(i\right)=\underset{S2\left(i\right)}{\arg \min }({\mathrm{\μ}\left|\right|S2\left(i\right)\left|\right|}_1+\frac{d}{2}{\left|\right|S2\left(i\right)+\frac{Q}{d}+S1\left(i\right)-S\left(i\right)\left|\right|}_F^2);$

Upgrade the Q value, update mode can but be not limited to: Q=Q+d (S1 (i)+S2 (i)-S (i));

After upgrading corresponding parameter S 1 (i), S2 (i) and Q, re-execute step (22);

(24) with current S1 (i) and the output of S2 (i) value, as the decomposition result for sparse part S, comprise feature S1 (i) and the impulsive noise S2 (i) of video itself.

Need to prove that above-mentioned steps (21) need to repeat N time to step (24), N is the frame number that video sequence comprises.Namely carry out N above-mentioned steps (21) to step (24), each self-corresponding video unique characteristics part of the sparse part of all frames in the video sequence and noise section will all be determined, at this moment, just obtain the decomposition result of the sparse part of all frames in the video sequence, that is: the S1 (1) that the 1st frame in the video sequence is corresponding and S2 (1), the S1 (2) that the 2nd frame in the video sequence is corresponding and S2 (2), the S1 (i) that i frame in the video sequence is corresponding and S2 (i) ... the S1 (N) that the N frame in the video sequence is corresponding and S2 (N).

(3) carry out the frame by frame operating process of video recovery for video sequence

In this process, the feature S1 (i) of the video that the low-rank part L that obtains according to process (1) and process (2) obtain itself carries out the noise removal process of video sequence, namely travel through video sequence i=1, N(is the N frame altogether), with feature S1 (i) addition of corresponding low-rank part L (i) and video itself, and the result who obtains after the addition is restricted to [P in the pixel value scope of permission _Min, P _Max], just can obtain to remove the i frame of the video sequence behind the noise; Wherein, for general video, [P _Min, P _Max] scope can be [0,255], corresponding L (i) has namely comprised the L that each frame is corresponding in the video sequence (i) value for to obtain among the low-rank part L from low-rank part L;

Particularly, suppose that M ' (i) is corresponding i the video pictures of removing behind the noise, i.e. video pictures corresponding to i frame, then M ' (i)=clip (P _Min, P _Max, L (i)+S1 (i)); The N frame that successively video sequence is comprised utilizes respectively this formula to carry out video recovery and processes the noise removal process that just can realize for whole video sequence, thereby obtains the vision signal behind the removal noise.

Can talk endlessly out by foregoing description, in the process of removing noise, not need the estimation of accurate noise intensity, the undesirable problem of noise remove effect that causes because of the noise intensity evaluated error so just can not occur.

Particularly, remove noise in the rgb video sequence as example to adopt the embodiment of the invention, specifically can be with the synthetic video of the sequence set of a plurality of scenes, and use Gaussian noise and the impulsive noise (salt-pepper noise and random impulsive noise) of varying strength that video is disturbed, form the video for the treatment of denoising.Noise size during the each test of note is (σ, p), and σ is from 5 to 40, the standard deviation of expression Gaussian noise; P from 5% to 45%, and expression is subjected to the pixel of impulse noise interference to account for the ratio of total pixel.And corresponding video is N=60, and namely continuous 60 frames carry out denoising as processing unit in each scene.Corresponding tol ₁=10 ^-1, tol ₂=10 ^-7, μ=2p+0.8.

Concrete experimental data such as following table:

In upper table, input represents to be subject to the mean P SNR(PeakSignaltoNoiseRatio of the video of noise jamming, Y-PSNR), output represents to adopt the mean P SNR of the video after the embodiment of the invention denoising.

Can find out by the experimental result in the upper table, through can obviously improving the PSNR value of video after the denoising of the embodiment of the invention, namely effectively improve the quality of video.

The below will illustrate the concrete adaptable scene of the embodiment of the invention.

(1) can be applied to the three primary colors for RGB() denoising of color image sequence

With reference to shown in Figure 4, for input be color video, and its form is the video of RGB, then at first scene detected, and video is divided into a plurality of scenes; Then to the video of each scene, be isolated into R, G, the video sequence of three color components of B fetches data according to the size of N to each video sequence, and the technical scheme of using the embodiment of the invention to provide is removed respectively noise; At last, with the R behind the removal noise, G, the three-component video sequence of B reconsolidates the formation color image sequence, reaches the purpose of video denoising.

(2) can be applied to the luminance and chrominance information for YUV() denoising of color image sequence

With reference to shown in Figure 5, for input be color video, and its form is the video of YUV, then at first converts every frame to rgb space from yuv space; Then, scene is detected, video is divided into a plurality of scenes; Then to the video of each scene, be isolated into R, G, the video sequence of three color components of B fetches data according to the size of N to each video sequence, and the technical scheme of using the embodiment of the invention to provide is removed respectively noise; Afterwards, with the R behind the removal noise, G, the three-component video sequence of B remerge and form the RGB color image sequence; At last, convert the rgb video sequence expression-form of YUV color space to, obtain the yuv video sequence after the denoising.

In the processing procedure that (three) can be applied to encode after the denoising for the RGB color image sequence

With reference to shown in Figure 6, the method that the embodiment of the invention provides or device can also be for the denoisings of the rgb video sequence of going the non-truck of editing and interviewing is gathered, afterwards, the rgb video sequence after the denoising can be sent to video coding apparatus and carry out Video coding compression processing.

In the processing procedure that (four) can be applied to show after the denoising for the RGB color image sequence

With reference to shown in Figure 7, the method that the embodiment of the invention provides or device can also be used for that video decode is arranged the rgb video sequence that obtains of decoding and carry out denoising, the rgb video sequence after the denoising can be play afterwards.

Certainly, the method that the embodiment of the invention provides or device can also be applied to other and need to carry out enumerating no longer one by one at this in the scene of denoising to video sequence.

The above; only for the better embodiment of the present invention, but protection scope of the present invention is not limited to this, anyly is familiar with those skilled in the art in the technical scope that the present invention discloses; the variation that can expect easily or replacement all should be encompassed within protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion with the protection range of claims.

Claims (6)

1. the denoising method of a video sequence is characterized in that, comprising:

According to constraints: $\begin{array}{c}\min {\left|\right|L\left|\right|}_{*}+\mathrm{\λ}{\left|\right|S\left|\right|}_1\\ s.t.L+S=M\end{array}$ Determine L and S, wherein, L is that low-rank part, S are sparse part, and M is video sequence, and λ is weight;

According to constraints: $\begin{array}{c}{\min \left|\right|S1\left(i\right)\left|\right|}_{\mathrm{TV}}+\mathrm{\μ}{\left|\right|S2\left(i\right)\left|\right|}_1\\ s.t.S1\left(i\right)+S2\left(i\right)=S\left(i\right)\end{array}$ Determine video unique characteristics part S1 (i) and the noise section S2 (i) of the sparse part correspondence of video sequence i frame, wherein, S (i) is described sparse part, and μ is weight;

After each self-corresponding video unique characteristics part of the sparse part of all frames in the video sequence and noise section all are determined, the decomposition result of the sparse part of all frames in the acquisition video sequence;

According to described low-rank part and the described video unique characteristics video sequence after definite denoising partly.

2. method according to claim 1 is characterized in that, the step of described definite L and S comprises:

Judge

Whether set up:

If set up, then calculate L and S according to following formula:

Also upgrade P:P=P+c (L+S-M), wherein, L=0 when initial, S=0, LaGrange parameter P=0, tol ₁=10 ^-2, c=1;

If be false, the L that then current calculating is obtained and S are as final calculation result.

3. method according to claim 1 and 2 is characterized in that, the step of described definite S1 (i) and S2 (i) comprises:

Judge

Whether set up:

If set up, then calculate S1 (i) and S2 (i) according to following formula:

Also upgrade Q:Q=Q+d (S1 (i)+S2 (i)-S (i)), wherein, S1 (i)=0 when initial, S2 (i)=0, LaGrange parameter Q=0, tol ₂=10 ^-7, d=1;

If be false, the S1 (i) that then current calculating is obtained and S2 (i) are as final calculation result.

4. the denoising device of a video sequence is characterized in that, comprising:

Video processing module is used for video sequence is decomposed into low-rank part and sparse part, concrete being used for according to the constraint bar

Part: $\begin{array}{c}\min {\left|\right|L\left|\right|}_{*}+\mathrm{\λ}{\left|\right|S\left|\right|}_1\\ s.t.L+S=M\end{array}$ Determine L and S, wherein, L is that low-rank part, S are sparse part, and M is video sequence, and λ is weight;

Sparse section processes module, the sparse decomposed that is used for described video processing module is obtained is the video unique characteristics

Part and noise section, concrete being used for according to constraints: $\begin{array}{c}{\min \left|\right|S1\left(i\right)\left|\right|}_{\mathrm{TV}}+\mathrm{\μ}{\left|\right|S2\left(i\right)\left|\right|}_1\\ s.t.S1\left(i\right)+S2\left(i\right)=S\left(i\right)\end{array}$ Determine video unique characteristics part S1 (i) and the noise section S2 (i) of the sparse part correspondence of video sequence i frame, wherein, S (i) is described sparse part, and μ is weight; After each self-corresponding video unique characteristics part of the sparse part of all frames in the video sequence and noise section all are determined, the decomposition result of the sparse part of all frames in the acquisition video sequence;

The denoising module is used for the partly video sequence after definite denoising of video unique characteristics that the low-rank part that obtains according to described video processing module and described sparse section processes module obtain.

5. device according to claim 4 is characterized in that, the step of described definite L and S comprises:

Judge

Whether set up:

If set up, then calculate L and S according to following formula:

Also upgrade P:P=P+c (L+S-M), wherein, L=0 when initial, S=0, LaGrange parameter P=0, tol ₁=10 ^-2, c=1;

If be false, the L that then current calculating is obtained and S are as final calculation result.

6. according to claim 4 or 5 described devices, it is characterized in that the step of described definite S1 (i) and S2 (i) comprises:

Judge Whether set up:

If set up, then calculate S1 (i) and S2 (i) according to following formula:

Also upgrade Q:Q=Q+d (S1 (i)+S2 (i)-S (i)), wherein, S1 (i)=0 when initial, S2 (i)=0, LaGrange parameter Q=0, tol ₂=10 ^-7, d=1;

If be false, the S1 (i) that then current calculating is obtained and S2 (i) are as final calculation result.

Images