CN105761251A - Separation method of foreground and background of video based on low rank and structure sparseness - Google Patents

Abstract

The present invention relates to a separation method of the foreground and the background of a video based on low rank and structure sparseness. The method comprises: the step 1, reading a video sequence; the step 2, performing low-rank and structure sparse matrix decomposition through adoption of a low-rank and structure sparse non-precision lagrangian multiplier method; the step 3, estimating the rank r of a background matrix A in advance; the step 4, reconstructing the background matrix A, namely, performing decomposition through a singular value, and calculating the background matrix A by using the rank r and a singular value contraction operator; and the step 5, reconstructing a foreground matrix E, obtaining the background matrix A through an observation matrix D and the step 4, and calculating a foreground matrix E; the step 6, determining the iteration times and the errors, and if the iteration times are equal to or smaller than iteration errors, the iteration is finished; and the step 7, recovering the background and the foreground images. The separation method of the foreground and the background of a video based on low rank and structure sparseness is able to directly realize the separation of the background and the foreground of the video sequence.

Description

A kind of video foreground background separating method sparse based on low-rank and structure

Technical field

The present invention relates to background extracting technology, in particular for the monitor video background under still camera, prospect isolation technics.

Background technology

Good prospect background separation scheme is one of key technology realizing moving object detection and identification.Background extracting is usually used in being partitioned into dynamic object from the scene that a still camera obtains, and typical method has: basic background modeling method, background estimating method, blurred background modeling and statistics background modeling method^[1].The basic ideas of these methods are to first pass through one section of training image sequential extraction procedures of study to go out the background characteristics of this video, thus setting up a mathematical model to describe its background, then by this background model, the video sequence needing detection is processed (being generally adopted background subtracting method), extract in present image with the pixel of different in kind in background model, be the dynamic object of image.Owing to the scene of video monitoring can vary over (illumination, shade etc.), these methods need the background model that upgrades in time, thus there is background model and can not adapt to the localized variation problem in scene rapidly and accurately.Simultaneously as need learning training sequence structure background model in advance, these all constrain they intelligent at video monitoring and in networking application.As can be seen here, to do not comprise the independent learning training stage and can accurately adapt to scene changes background extracting method research tool be of great significance.Rank of matrix is estimated by low-rank matrix decomposition as one is sparse, it is possible to is effectively focused to find out its low-dimensional eigenspace from the observation data by strong noise pollution or partial loss, and recovers original observation signal.In video, the background parts of each frame is only limited by a small amount of controlling factors, thus showing the characteristic of low-rank；And moving target or prospect can detect by identifying the residual error of space sparse distribution, so video sequence meets low-rank and adds sparse structure, can as low-rank matrix resolution problem^[2]。

The openness of signal is not that unique signal represents model, and imperfection is gone back in low-rank matrix decomposition in theory, shows that nuclear norm can not completely correctly approach rank of matrix function^[3], signal self yet suffers from some structure prior informations and is not excavated completely^[4]。

List of references:

[1]BouwmansT.,BafF.E.,VachonA.B.StatisticalbackgroundModelingforforegrounddetection:Asurvey[J].HandbookofPatternRecognitionandComputerVision,2010,4(2):18-189.

[2]ChengL.,GongM.,SchuurmansD.,CaelliT.Realtimediscriminativebackgroundsubtraction[J].IEEETransactionsonImageProcessing,2011,20(5):1401–1404.

[3]HuY.,ZhangD.,YeJ.,LiX.,HeX.FastandAccurateMatrixCompletionviaTruncatedNuclearNormRegularization[J].IEEETransactionsonPatternAnalysisandMachineIntelligence,2013,35(9):2117-2130.

[4] Peng Yigang, Suo Jinli, wears Qionghai etc. and sense low-rank matrix from compression and recover: theoretical and application [J]. automatization's journal, 2013,39 (7): 981-994..

[5]DuarteM.,EldarY.C.Structuredcompressedsensing:fromtheorytoapplication[J].IEEETransactionsonSignalProcessing,2011,59(9):4053-4085..

[6]YangJ.,YinW.,ZhangY.,WangY.AFastAlgorithmforEdgePreservingVariationalMultichannelImageRestoration[J].SIAMJournalonImagingSciences,2009,2(2):569-592.

[7]CaiJ.,CandesE.,ShenZ.Asingularvaluethresholdingalgorithmformatrixcompletion[J].SIAMJournalonOptimization,2010,20:1956-1982.

[8]StatisticalModelingofComplexBackgroundforForegroundObjectDetection(videodemo),[Online].Available:http://perception.i2r.astar.edu.sg/bk_model/bk_index.html.

[9]LinZ.,ChenM.,WuL.TheaugmentedLagrangemultipliermethodforexactrecoveryofcorruptedlow-rankmatrices[R].TechnicalReportUILU-ENG-09-2215,Univ.Illinois,Urbana-Champaign,Oct.2009.

[10]AybatN.,IyengarG.AnAlternatingDirectionMethodwithIncreasingPenaltyforStablePrincipalComponentPursuit,[Online].Available:http://arxiv.org/1309.6553.

Summary of the invention

For the shortcoming of tradition background extracting method, in conjunction with the prospect priori of monitor video, it is proposed to a kind of matrix decomposition algorithm sparse based on low-rank and structure, it is directly realized by monitor video sequence the separation of background and prospect.Technical scheme is as follows:

A kind of video foreground background separating method sparse based on low-rank and structure, comprises the following steps:

Step 1: read in video sequence

Video comprises n frame image sequence, every color image frame is converted to gray level image, and is that a m ties up row vector, i.e. m=n1 × n2 by the graphical representation that every frame dimension is n1 × n2, n frame video image reads in and arranges formation observing matrix D ∈ R successively^m×n。

Step 2: adopting low-rank and the sparse non-precision method of Lagrange multipliers of structure to carry out the matrix decomposition sparse based on low-rank and structure, parameter is provided that

Take regularization parameterY₀=D/J (D), μ₀=1.25/ | | D | |₂, wherein, J (D)=max (| | Y | |₂, | | Y | |_∞), | | | |_∞Value maximum in representing matrix element absolute value, regularization parameter β span (0,0.1)；

Step 3: the order r of pre-estimation background matrix A；

Step 4: rebuild background matrix A:

By singular value decomposition, order r and singular value contraction operator calculating background matrix A, the iterative formula calculating background matrix A is:Wherein E₀=0, Y ∈ R^m×nFor linear equality constraints multiplier, μ > 0 it is little positive number, represent the penalty factor being unsatisfactory for linear equality constraints, subscript k represents current iteration number of times；

Step 5: rebuild prospect matrix E:

Calculated by observing matrix D, step 4 and obtain background matrix A, calculate prospect matrix E；

Step 6: judge iterations and error, if reaching iterations or less than iteration error, then stops iteration, otherwise, returns step 4 and continues iteration；

Step 7: restore background and foreground image:

Take calculated background matrix A and the column vector of prospect matrix E, and be reduced to n respectively₁×n₂Background image and foreground image, recover observation background corresponding to each frame of video and mobile target image.

Adopt the method for the invention, compared with tradition background extracting technology, it is possible to be directly realized by monitor video sequence the separation of background and prospect, hence it is evident that the prospect that improves rebuilds image visual quality, saves the time of process, has reached the effect close to practicality.

Accompanying drawing explanation

Fig. 1 institute of the present invention extracting method block diagram

Fig. 2 institute of the present invention extracting method background extracting result example, (a) Lobby sequence (b) Bootstrap sequence

Detailed description of the invention

Fig. 1 show the block diagram of institute of the present invention extracting method.Institute's extracting method supposes that pending monitor video sequence is shot by still camera.Each two field picture is expressed as a m and ties up row vector v_iIf this video comprises n frame image sequence altogether, then this observation video just can use the data matrix D=[v of n row vector composition₁, v₂... v_i…]∈R^m×n(i=1,2 ..., n) represent.Representing video background part by matrix A, E represents video foreground part simultaneously, then can obtain following model:

D=A+E, (1)

Ideally, will not there is significant change in the background image of monitor video, thus the order of matrix A is far smaller than min, and (m, n) (close to 1).Solved by effective low-rank matrix decomposition algorithm, calculate A and E, it is possible to isolate background and moving target.The algorithm model of the present invention is as follows:

$\begin{array}{ccc}\underset{A,E}{min}\left|\right|A|{|}_{*}+\mathrm{\α}|\left|E\right|{|}_1+\mathrm{\β}\left|\right|E|{|}_{1,2}& s.t.& D=A+E.\end{array}---\left(2\right)$

In formula, | | | | * represents nuclear norm (singular values of a matrix sum), | | | | 1 represents l₁Norm (in matrix all elements absolute value sum).

For the sparse induced norm of structure (StructuredSparsity-InducingNorms), [E_ij] pixel value of the i-th row j row place element in representing matrix E.l_1,2Norm utilizes the space of nonzero element and local prior information in matrix, and in each row of induced matrix E, element pixel value is 0, but does not affect for nonzero element pixel value in non-zero row.α, β are the parameter more than 0, for balancing the low-rank degree of A and the sparse degree of E, it is possible to according to l₁Norm, l_1,2The characteristic of norm or experience arrange the size of α, β.

The present invention adopts augmented vector approach (ALM)^[9]Etc. solving the matrix decomposition problem sparse based on low-rank and structure.Augmented Lagrangian Functions based on the sparse matrix decomposition model (2) of low-rank and structure is:

$L(A,E,Y,U)=\left|\right|A|{|}_{*}+\mathrm{\&alpha;}|\left|E\right|{|}_1+\mathrm{\&beta;}\left|\right|E|{|}_{1,2}+<Y,D-A-E>+\frac{\mathrm{\&mu;}}{2}||Y-A-E|{|}_F^2,---\left(3\right)$

In formula, Y ∈ R^m×nFor linear equality constraints multiplier, the mark (matrix exgenvalue summation) of < > representing matrix, μ > 0 is smaller positive number, represents the penalty factor being unsatisfactory for linear equality constraints, | | | |_FRepresent Frobenius norm (matrix element quadratic sum opens radical sign again).ALM algorithm adopts the method alternately updated to come solution matrix A, E and Y.Alternately the main thought of update method is that required variable is divided into different blocks.When solving the minimization problem of variable in certain block, in other blocks, variable value remains unchanged.ALM algorithm needs to solve the solution of following three minimization problem:

A_k+1=argmin_A(A,E_k,Y_k；μ),

E_k+1=argmin_E(A_k+1,E,Y_k；μ),(4)

Y_k+1=Y_k+μ(D-A_k+1-E_k+1),

For first minimization problem in formula (4), its fixing E and Y seeks an A making formula (3) minimize, it may be assumed that

$\underset{A}{min}\left|\right|A|{|}_{*}+\frac{\mathrm{\&mu;}}{2}||(D-E_k+{\mathrm{\&mu;}}^{-1}{Y}_k)-A|{|}_F^2,---\left(5\right)$

Formula (5) can be solved by singular value contraction operator, it may be assumed that

$A_{k+1}=D_{{\mathrm{\&mu;}}^{-1}}(D-E_k+{\mathrm{\&mu;}}^{-1}{Y}_k),---\left(6\right)$

The definition of singular value contraction operator is as follows:

If matrixBe an order being the matrix of r, its singular value decomposition (SVD) is: X=U Σ V^T, $U\&Element;R^{n_1\&times;r},\mathrm{\&Sigma;}=diag\left({\left\{{\mathrm{\&sigma;}}_i\right\}}_{i=1}^r\right)\&Element;R^{r\&times;r},V\&Element;R^{n_2\&times;r}.$

Then D_τ(X)=UD_τ(Σ)V^T _, $D_{\mathrm{\&tau;}}\left(\mathrm{\&Sigma;}\right)=diag\left({\{{\mathrm{\&sigma;}}_i-\mathrm{\&tau;}\}}_{i=1}^r\right).$

For second minimization problem in formula (4), its fixing A and Y seeks an E making formula (3) minimize, it may be assumed that

$\underset{E}{min}\mathrm{\&alpha;}\left|\right|E|{|}_1+\mathrm{\&beta;}|\left|E\right|{|}_{1,2}+\frac{\mathrm{\&mu;}}{2}\left|\right|(D-A_{k+1}+{\mathrm{\&mu;}}^{-1}{Y}_k)-E|{|}_F^2,---\left(7\right)$

Formula (7) had both contained l₁Norm contains again l_1,2Norm.Being inspired by soft-threshold operator, the present invention adopts soft-threshold operation operator that a kind of row shrinks to solve formula (7), and this contraction operator is defined as:

Wherein,

When then updating E:

E_k+1=S_1,1,2(D-A_k+1+μ^-1Y_k, α/μ, β/μ), (8)

The details of ALM algorithm are such as shown in algorithm 1.

Step 1: read in video sequence

For certain class by the acquired n frame monitor video sequence of still camera, first every color image frame is converted to gray level image, and is n by every frame dimension₁×n₂Graphical representation be that m ties up row vector (i.e. m=n₁×n₂), n frame video image reads in and arranges formation observing matrix D ∈ R successively^m×n。

Step 2: algorithm parameter is arranged.

Algorithm 1 takes regularization parameterY₀=D/J (D), μ₀=1.25/ | | D | |₂.Wherein, J (D)=max (| | Y | |₂, | | Y | |_∞), | |_∞Value maximum in representing matrix element absolute value.Regularization parameter β span (0,0.1).

Step 3: the order pre-estimation of background matrix A.

Estimate the order r of A.

Step 4: rebuild background matrix A.

By singular value decomposition, order r and singular value contraction operator calculate background matrix A.

Step 5: rebuild prospect matrix E.

Calculated by observing matrix D, step 4 and obtain background matrix A, pass through E_k+1=S_1,1,2(D-A_k+1+μ^-1Y_k, α/μ, β/μ) calculate prospect matrix E.

Step 6: judge iterations and error.

When iterations is less than 1000 and iteration error | | D-A_k-E_k||_F/||D||_FMore than 1.0 × 10^-6Time, algorithm 1 continues iteration, repeats step 3；When iterations is more than 1000, or iteration error | | D-A_k-E_k||_F/||D||_FLess than 1.0 × 10^-6Time, algorithm 2 stops iteration, performs step 7.

Step 7: restore background and foreground image

Take calculated background matrix A and the column vector of prospect matrix E, and be reduced to n respectively₁×n₂Background image and foreground image, recover observation background corresponding to each frame of video and mobile target image.

Adopt the matlab2013b under Windows7SP1 system as experiment simulation platform.Select 300 frame Lobby video images, 250 frame Bootstrap video images as test set^[8].The rate respectively of the every two field picture of Lobby sets of video data is 168 × 120, and the resolution of the every two field picture of Bootstrap sets of video data is 160 × 120.Every frame gray level image is become a column vector, the arrangement of each frame column vector the observing matrix of Lobby and the Bootstrap sets of video data formed D respectively₀(20160×300)、D₁(19200×250).Adopt method and the IALM of present invention proposition^[9]、ADMIP^[10]Test video is carried out background extracting, obtains good treatment effect.Fig. 2 (a) is 3 kinds of algorithm background extracting contrast effect figure of 2 two field pictures that 3 kinds of algorithm background extracting contrast effect figure, Fig. 2 (b) are the Bootstrap sequence chosen respectively of 2 two field pictures of the Lobby sequence chosen respectively.In each figure, the left side is original test video image, the middle low-rank background image for recovering, and the right is the sparse foreground image recovered, and is followed successively by IALM, ADMIP and the algorithm restoration result of present invention proposition from top to bottom.

Claims (1)

1., based on the video foreground background separating method that low-rank and structure are sparse, comprise the following steps:

Step 1: read in video sequence

Video comprises n frame image sequence, every color image frame is converted to gray level image, and is n by every frame dimension₁×n₂Graphical representation be that m ties up row vector, i.e. m=n₁×n₂, n frame video image reads in and arranges formation observing matrix D ∈ R successively^{m ×n}。

Step 2: adopting low-rank and the sparse non-precision method of Lagrange multipliers of structure to carry out the matrix decomposition sparse based on low-rank and structure, parameter is provided that

Take regularization parameterY₀=D/J (D), μ₀=1.25/ | | D | |₂, wherein, J (D)=max (| | Y | |₂, | | Y | |_∞), | | | |_∞Value maximum in representing matrix element absolute value, regularization parameter β span (0,0.1)；

Step 3: the order r of pre-estimation background matrix A；

Step 4: rebuild background matrix A:

By singular value decomposition, order r and singular value contraction operator calculating background matrix A, the iterative formula calculating background matrix A is:Wherein E₀=0, Y ∈ R^m×nFor linear equality constraints multiplier, μ > 0 it is little positive number, represent the penalty factor being unsatisfactory for linear equality constraints, subscript k represents current iteration number of times；

Step 5: rebuild prospect matrix E:

Calculated by observing matrix D, step 4 and obtain background matrix A, calculate prospect matrix E；

Step 6: judge iterations and error, if reaching iterations or less than iteration error, then stops iteration, otherwise, returns step 4 and continues iteration；

Step 7: restore background and foreground image:

Take calculated background matrix A and the column vector of prospect matrix E, and be reduced to n respectively₁×n₂Background image and foreground image, recover observation background corresponding to each frame of video and mobile target image.