CN106204477B - Video frequency sequence background restoration methods based on online low-rank background modeling - Google Patents

Video frequency sequence background restoration methods based on online low-rank background modeling Download PDF

Info

Publication number
CN106204477B
CN106204477B CN201610526248.0A CN201610526248A CN106204477B CN 106204477 B CN106204477 B CN 106204477B CN 201610526248 A CN201610526248 A CN 201610526248A CN 106204477 B CN106204477 B CN 106204477B
Authority
CN
China
Prior art keywords
background
frame
video
matrix
indicate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201610526248.0A
Other languages
Chinese (zh)
Other versions
CN106204477A (en
Inventor
杨敬钰
杨蕉如
杨雪梦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin University
Original Assignee
Tianjin University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin University filed Critical Tianjin University
Priority to CN201610526248.0A priority Critical patent/CN106204477B/en
Publication of CN106204477A publication Critical patent/CN106204477A/en
Application granted granted Critical
Publication of CN106204477B publication Critical patent/CN106204477B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The invention belongs to video analysis fields, to realize that the background to video sequence carries out accurately online recovery.The technical solution adopted by the present invention is that; video frequency sequence background restoration methods based on online low-rank background modeling; the low-rank matrix that nuclear norm is introduced on the basis of traditional background modeling is decomposed and the motion information of binaryzation is estimated, to solve the problems, such as that prior art can not be handled.The present invention includes the following steps: 1) to be specifically expressed as background recovery problem in video sequence to solve following unconstrained optimization equation: 2) constructing kth frame movement mapping weight vectors wk: 3) it is solved using alternating direction method: 4) being solved5) it solves6) it repeats the above steps 4), 5) until algorithmic statement；7) it is solved using substitution of variable: 8) being solved9) it solves10) it repeats the above steps 8), 9) until algorithmic statement；11) video background is solved.Present invention is mainly applied to video analysis occasions.

Description

Video frequency sequence background restoration methods based on online low-rank background modeling
Technical field
The invention belongs to video analysis fields.In particular to the video frequency sequence background based on online low-rank background modeling restores Method.
Background technique
With the fast development of internet and mass media, the availability of video data increases sharply, far beyond people Manual analyzing range.Therefore, interested information is excavated in a large amount of video using automatic video frequency analysis has extremely Important meaning.Background recovery is widely used in many views as a preconditioning technique for extracting interesting target in video Feel in application, such as target detection, target following and Activity recognition.
In recent years, there are many research achievements about the method for solving the problems, such as video background recovery, it is a kind of most important Method is the method based on Robust Principal Component Analysis (RPCA), the general idea is that: it is assumed that being linear phase between video background frame Close, video is resolved into low-rank background and sparse prospect two parts, wherein for low-rank background approximation using nuclear norm come Constraint.Such methods can be realized simultaneously the recovery of video background and the detection of sport foreground.But it solves nuclear norm to require pair All video frames carry out singular value decomposition simultaneously, and therefore, limitation the method can only be solved in a manner of batch processing, Wu Fashi Now real-time background recovery.In addition, every iteration is primary when solving such method, require at all video frames Reason, causes algorithm time and memory efficient lower, is not suitable for stream-type video and big video.So in the epoch of big data Under background, it is very necessary for finding a kind of effective online background recovery method.
At this stage, for above-mentioned traditional batch processing RPCA method the shortcomings that, some online back have been proposed both at home and abroad Scape restoration methods, it can processing in real time is carried out to single frame video image and obtains its corresponding background.Its thinking is: to limitation The nuclear norm item of batch processing implementation carries out low-rank matrix decomposition, obtains the base and coefficient corresponding with base phase of background.Herein On the basis of, the decomposition of matrix low-rank sparse is carried out to each frame video image respectively, the background finally restored is to solve obtained base With the product of coefficient transposition.This kind of online restoration methods can restore the background of each frame image in real time, greatly improve The memory efficient of algorithm and the applicability of streaming data.But because it ignores the motion information of video, in prospect Slow region is moved, smear effect is easy to appear in the background of recovery.To overcome this disadvantage, the present invention uses in a model Optical flow algorithm estimates motion information.
Summary of the invention
The invention is intended to make up the deficiencies in the prior art, that is, realize the background of video sequence is carried out it is accurately online extensive It is multiple.The technical solution adopted by the present invention is that the video frequency sequence background restoration methods based on online low-rank background modeling, traditional The low-rank matrix that nuclear norm is introduced on the basis of background modeling is decomposed and the motion information of binaryzation is estimated, to solve prior art The problem of can not handling.The present invention includes the following steps:
1) background recovery problem in video sequence is specifically expressed as solving following unconstrained optimization equation:
Wherein | | | |FThis black norm of the not Luo Beini of representing matrix, | | | |*The nuclear norm of representing matrix, | | | |1 One norm of representing matrix, ο indicate that the point multiplication operation of two matrixes, D are that actual video sequence frame successively presses column vector arrangement Made of matrix, B indicates background to be restored, and E represents foreground part, and W indicates that two-value sports ground maps weight matrix, λ1, λ2Point Not Biao Shi in video low-rank background and prospect weight coefficient；
To solve nuclear norm needs singular value decomposition SVD is carried out to all frames together, low-rank square is carried out to background B to be restored Battle array is decomposed:
Wherein inf { } expression takes infimum to { }, and L is the base of the corresponding lower-dimensional subspace of video frequency sequence background, and C is Video sequence corresponds to the coefficient matrix of base L；Obtain following decomposition model:
When solving this equation, using alternate optimization method, following loss function is minimized with incremental mode:
Wherein n indicates the current totalframes of video sequence, dkIndicate the column vector of video sequence kth frame,It is K frame image dkThe corresponding loss function at base L, is defined as follows:
Wherein | | | |2Indicate two norms of vector, w indicates present frame dkMovement map weight vectors, c is dkIn base L Under coefficient vector, e be present frame foreground part；
Solution procedure is, online to each frame image dkSuccessively: estimation motion information calculates kth frame movement mapping weight Vector wk, update kth frame coefficient vector ck, update kth frame prospect ek, the corresponding base L of k frame before updatingk；
2) building kth frame movement mapping weight vectors wk: w is solved using backward motion estimation scheme onlinek；
It is based on sparse prospect it is assumed that two-value movement weight is all assigned a value of 1 to first frame；For subsequent frame, respectively It is then that movement is reflected by its binaryzation using the dense sports ground of optical flow algorithm estimation present frame using first frame as reference frame It penetrates；
3) equation (5) following sequence is converted into using alternating direction method ADM to solve:
In above formulaIndicate the value of variable c when being minimized objective function,Expression makes The value of variable e when objective function is minimized, l are the number of iterations；Then method according to step 4), 5) is iterated solution Obtain coefficient vector ckWith prospect ek；
4) it solvesClosed solutions by solving least square problem acquire coefficient
Remove and solves kth frame coefficient vector c in formula (7)kObjective function in ckUnrelated item obtains following equation:
It is acquired using least square methodSolution are as follows:
WhereinIndicate movement mapping wkDiagonal matrix, i.e., by vector wkEach element be successively put into matrix's On leading diagonal, I indicates unit matrix；
5) it solvesKth frame prospect is acquired using contraction operator
Remove and solves kth frame prospect e in formula (7)kObjective function in ekUnrelated item obtains following equation:
Formula (10) is solved using contraction operator:
Wherein eik, sik, wikRespectively indicate ek, sk, wkIth pixel value,
6) repeat the above steps 4), 5) until algorithmic statement, the at this moment result of iterationWithIt is exactly present frame dkBe Number vector ckWith prospect ek, byThe coefficient matrix C of k frame image before obtainingk；
7) equation (3) following unconstrained optimization problem is converted to using substitution of variable to solve:
WhereinFor the variable newly introduced, Rk=[r1,...,rk] indicate the present frame that has acquired and The background of video frame before, rk=dk-ekIndicate the background of the kth frame image acquired, λ3For weight coefficient；
Equation (12) following sequence is converted into using alternating direction method ADM to solve:
Then method according to step 8), 9) is iterated solution and obtains variable YkWith the base of video sequence lower-dimensional subspace Lk；
8) it solvesMode acquires pixel-by-pixel
Remove and solves Y in formula (13)kObjective function in YkUnrelated item obtains following equation:
Using least square method, calculates solve pixel-by-pixel:
WhereinIndicate base LkThe i-th row, yikIt indicatesThe element value of i-th row kth column, rikIndicate rkIth pixel Value；
9) it solvesClosed solutions by solving least square problem acquire base
Remove and solves L in formula (13)kObjective function in LkUnrelated item obtains following equation:
It is acquired using least square methodSolution are as follows:
Further, intermediate variable P is introducedk=RkCk,It is efficiently solved with incremental mode
Wherein:
10) repeat the above steps 8), 9) until algorithmic statement, the at this moment result of iterationExactly preceding k frame image background Base Lk；
6) and 10) 11) video background B is solved: by the C respectively obtainedkAnd LkIt acquiresBkExactly former problem Last solution B.
The specific formula of step 2) is:
Wherein wikIndicate that the movement of kth frame image ith pixel maps weight, as wkIth pixel value,WithThe horizontal motion components and vertical motion component of kth frame image ith pixel are respectively indicated, τ is for binaryzation sports ground Threshold value.
Technical characterstic and effect of the invention:
The method of the present invention restores problem for the background of video sequence online, by introducing the low-rank matrix point to nuclear norm Solution and the estimation frame by frame to video motion information, realize the solution for restoring problem online to the background of video sequence.The present invention It has the following characteristics that
1, the advantages of having used alternating direction method (ADM) algorithm, contraction operator to solve subproblem, incorporate existing algorithm.
2, the online recovery to video frequency sequence background is realized.Core is introduced in traditional matrix low-rank sparse decomposition model The low-rank decomposition of norm can carry out online recovery to background, greatly improve memory efficient, be suitable for streaming video It is handled with big video data.
3, low-rank matrix is decomposed and motion information estimation combines, the movement letter being introduced into optical flow algorithm estimation video Breath, and be movement weight matrix by its binaryzation, the sport foreground in video background can be more effectively separated, so that video sequence Column background recovery result is more accurate.
Detailed description of the invention
Fig. 1 is algorithm flow chart；
Fig. 2 is the background recovery result figure of original browser1 video frame and use the method for the present invention；
Fig. 3 is the background recovery result figure of original browser1 video frame and use the method for the present invention.
Specific embodiment
Low-rank decomposition and the motion information estimation of nuclear norm are introduced in traditional matrix low-rank sparse decomposition model, so that The accurate background of video sequence, i.e. the video frequency sequence background recovery side based on online low-rank background modeling can be recovered online Method, to solve the problems, such as that prior art can not be handled.It elaborates with reference to the accompanying drawings and examples to the present invention.
1) background recovery problem in video sequence is specifically expressed as solving following unconstrained optimization equation:
Wherein | | | |FThis black (Frobenius) norm of the not Luo Beini of representing matrix.||·||*The core model of representing matrix Number.||·||1One norm of representing matrix.ο indicates the point multiplication operation of two matrixes.D is that actual video sequence frame is successively pressed Matrix made of column vector arrangement.B indicates background to be restored.E represents foreground part.W indicates that two-value sports ground maps weight Matrix.λ1, λ2Respectively indicate the weight coefficient of low-rank background and prospect in video.
11) all frames are carried out with singular value decomposition (SVD) together to solve nuclear norm needs, the present invention is to background to be restored B carries out low-rank matrix decomposition:
Wherein inf { } indicates to take { } infimum, and L is the base of lower-dimensional subspace, and C is that video frame corresponds to base L Coefficient, CTThe transposition of representing matrix C.The matrix low-rank sparse that model (1) is converted into following online motion information auxiliary is decomposed Model:
12) when solving this equation, the present invention uses alternate optimization method, minimizes following loss function with incremental mode:
Wherein n indicates the current totalframes of video sequence, dkIndicate the column vector of video sequence kth frame,For kth The corresponding loss function of frame image, is defined as follows:
Wherein | | | |2Two norms of representing matrix.W indicates present frame dkMovement map weight vectors.C is dkIn base L Under coefficient vector.E is the foreground part of present frame.
13) solution procedure is, online to each frame image dkSuccessively: estimation motion information calculates kth frame movement mapping Weight vectors wk, update kth frame coefficient vector ck, update kth frame prospect ek, the corresponding base L of k frame before updatingk。
2) building kth frame movement mapping weight vectors wk: w is solved using backward motion estimation scheme onlinek；
21) based on sparse prospect it is assumed that two-value movement weight is all assigned a value of 1 to first frame.
22) for subsequent frame, respectively using first frame as reference frame, using the dense movement of optical flow algorithm estimation present frame , it is then movement mapping by its binaryzation, specific formula is as follows:
Wherein wikIndicate that the movement of kth frame image ith pixel maps weight, as wkIth pixel value,WithRespectively indicate the horizontal motion components and vertical motion component of kth frame image ith pixel.τ is for binaryzation sports ground Threshold value, according to the average pixel value of the sports ground of estimation set.
3) present invention solves equation (5) using alternating direction method (ADM), i.e., equation (5) is converted into following sequence and carried out It solves:
In above formulaIndicate the value of variable c when being minimized objective function,Expression makes The value of variable e when objective function is minimized.L is the number of iterations.Set each initial parameter values, then according to step 4), 5)
Method be iterated solution, obtain the corresponding coefficient vector c of kth frame imagekWith prospect ek。
4) it solvesClosed solutions by solving least square problem acquire kth frame coefficient
Remove and solves coefficient c in formula (7)kObjective function in ckUnrelated item obtains following equation:
The corresponding base L of k frame image before givenkWith kth frame prospectIt is acquired using least square methodSolution are as follows:
WhereinIndicate the movement map vector w of kth frame imagekDiagonal matrix, i.e., by vector wkEach element according to It is secondary to be put into matrixLeading diagonal on,Other positions element value be 0.I indicates unit matrix.
5) it solvesKth frame prospect is acquired using contraction operator
Remove and solves e in formula (7)kObjective function in ekUnrelated item obtains following equation:
Given base LkWith iteration result 4)Formula (10) is solved using contraction operator:
Wherein eik, sik, wikRespectively indicate ek, sk, wkIth pixel,
6) repeat the above steps 4), 5) until algorithmic statement, the at this moment result of iterationWithIt is exactly present frame dkBe Number vector ckWith prospect ek.It willIt is assigned to the row k of C, the coefficient C of k frame image before then obtainingk；
7) substitution of variable and alternating direction method (ADM) solving model (3) are used.
71) because of the base L of direct solution backgroundkIt needs to carry out derivation to matrix dot product, is not easy to solve, so the present invention makes Equation (3) following unconstrained optimization problem is converted to substitution of variable to solve:
WhereinIt is for convenience of the variable for calculating and newly introducing, Rk=[r1,...,rk] indicate to have asked The background of the preceding k frame obtained, rk=dk-ekIndicate the background of the kth frame image acquired, λ3For weight coefficient.
72) equation (12) following sequence is converted into using alternating direction method (ADM) to solve:
Set variable YkInitial value, then method according to step 8), 9) is iterated solution and obtains variableAnd view The base L of frequency sequence lower-dimensional subspacek。
8) it solvesRemove and solves Y in formula (13)kObjective function in YkUnrelated item obtains following equation:
Using least square method, calculates solve pixel-by-pixel:
WhereinIndicate base LkThe i-th row, yikIt indicatesThe element value of i-th row kth column, rikIndicate rkIth pixel Value.
9) it solvesClosed solutions by solving least square problem acquire base
91) remove and solve L in formula (13)kObjective function in LkUnrelated item obtains following equation:
It is acquired using least square methodSolution are as follows:
92) in order to improve memory efficient, the present invention is further introduced into intermediate variable Pk=RkCk,It is efficiently solved with incremental modeEquation is as follows:
Wherein intermediate variable Pk、XkAnd ZkUpdate rule are as follows:
10) repeat the above steps 8), 9) until algorithmic statement, the at this moment result of iterationExactly preceding k frame image background Base Lk；
6) and 10) 11) the background B of video sequence is solved: by the C respectively obtainedkAnd LkIt acquiresBkIt is exactly former Problem last solution B.
The method of the present invention decomposes low-rank matrix and motion information estimation combines, and decomposes in traditional matrix low-rank sparse The low-rank decomposition of nuclear norm and the motion information weight of two-value are introduced in model, so that solve that prior art can not handle asks Topic is realized and carries out accurately restoring (experiment flow figure is as shown in Figure 1) online to video frequency sequence background.In conjunction with attached drawing and implementation Detailed description are as follows for example:
1) the browser1 video frame (as shown in Figure 2) of 115 × 86 pixels is used to input in experiment as initial data, often One frame image slices vegetarian refreshments is m=115 × 86.Original video frame is processed into column vector form and handled by the present invention, by it It is arranged in order composition matrix D, restores the video frequency sequence background problem and is specifically expressed as solving following unconstrained optimization equation:
Wherein | | | |FThis black (Frobenius) norm of the not Luo Beini of representing matrix.||·||*The core model of representing matrix Number.||·||1One norm of representing matrix.ο indicates the point multiplication operation of two matrixes.B indicates background to be restored.Before E is represented Scape part.W indicates that two-value sports ground maps weight matrix.λ1, λ2Respectively indicate the weight system of low-rank background and prospect in video Number, value is respectively in an experiment1。
11) need to carry out all frames together singular value decomposition to solve nuclear norm, the present invention carries out low-rank matrix point to B Solution:
Wherein inf { } indicates to take { } infimum, and L is the base of lower-dimensional subspace, and C is that video frame corresponds to base L Coefficient, CTThe transposition of representing matrix C.The matrix low-rank sparse that model (1) is converted into following online motion information auxiliary is decomposed Model:
The order value of L is 5 in experiment.
12) when solving this equation, the present invention uses alternate optimization method, minimizes following loss function with incremental mode:
Wherein n indicates the current totalframes of video sequence, dkIndicate the column vector of video sequence kth frame,For kth The corresponding loss function of frame image, is defined as follows:
Wherein | | | |2Two norms of representing matrix.W indicates present frame dkMovement map weight vectors.C is dkIn base L Under coefficient vector.E is the foreground part of present frame.
13) solution procedure is, online to each frame image dkSuccessively: estimation motion information calculates kth frame movement mapping Weight vectors wk, update kth frame coefficient vector ck, update kth frame prospect ek, the corresponding base L of k frame before updatingk。
2) building kth frame movement mapping weight vectors wk: w is solved using backward motion estimation scheme onlinek。
21) based on sparse prospect it is assumed that two-value movement weight is all assigned a value of 1 to first frame.
22) for subsequent frame, respectively using first frame as reference frame, using the dense movement of optical flow algorithm estimation present frame , it is then movement mapping by its binaryzation, specific formula is as follows:
Wherein wikIndicate that the movement of kth frame image ith pixel maps weight, as wkIth pixel value,WithRespectively indicate the horizontal motion components and vertical motion component of kth frame image ith pixel.τ is for binaryzation sports ground Threshold value, set according to the average pixel value of the sports ground of estimation, take τ=0.1 in experiment.
3) present invention solves equation (5) using alternating direction method (ADM), i.e., equation (5) is converted into following sequence and carried out It solves:
In above formulaIndicate the value of variable c when being minimized objective function.Expression makes The value of variable e when objective function is minimized.L is the number of iterations.Set each initial parameter values, then according to step 4), 5) Method be iterated solution, obtain coefficient vector ckWith prospect ek.Initialization in experiment are as follows: l=1；K=1；w1=1；L1The preceding 5 principal component assignment obtained after singular value decomposition are done by ten frame background frames of the video sequence.
4) correspond to base L by solving the closed solutions of least square problem and solvingkCoefficient ck。
Remove and solves coefficient c in formula (7)kObjective function in ckUnrelated item obtains following equation:
The corresponding base L of k frame image before givenkAnd prospectIt is acquired using least square methodSolution are as follows:
WhereinIndicate movement mapping wkDiagonal matrix, i.e., by vector wkEach element be successively put into matrix's On leading diagonal,Other positions element value be 0.I indicates unit matrix.
5) it solvesKth frame prospect is acquired using contraction operator
Remove and solves e in formula (7)kObjective function in ekUnrelated item obtains following equation:
Given base LkWith iteration result 4)Formula (10) is solved using contraction operator:
Wherein eik, sik, wikRespectively indicate ek, sk, wkIth pixel,
6) repeat the above steps 4), 5) until algorithmic statement, the at this moment result of iterationWithIt is exactly present frame dkBe Number vector ckWith prospect ek.It willIt is assigned to the row k of C, the coefficient C of k frame image before then obtainingk。
7) substitution of variable and alternating direction method (ADM) solving model (3) are used.
71) because of the base L of direct solution backgroundkIt needs to carry out derivation to matrix dot product, is not easy to solve, so the present invention makes Equation (3) following unconstrained optimization problem is converted to substitution of variable to solve:
WhereinIt is for convenience of the variable for calculating and newly introducing, Rk=[r1,...,rk] indicate to have asked The background of the preceding k frame obtained, rk=dk-ekIndicate the background of the kth frame image acquired, λ3For weight coefficient.
72) equation (12) following sequence is converted into using alternating direction method (ADM) to solve:
Variable YkInitial value be set as 0, then method according to step 8), 9) is iterated solution and obtains variableAnd view The base L of frequency sequence lower-dimensional subspacek。
8) it solvesRemove and solves Y in formula (13)kObjective function in YkUnrelated item obtains following equation:
Using least square method, calculates solve pixel-by-pixel:
WhereinIndicate base LkThe i-th row, yikIt indicatesThe element value of i-th row kth column, rikIndicate rkIth pixel Value.
9) it solvesClosed solutions by solving least square problem acquire base
91) remove and solve L in formula (13)kObjective function in LkUnrelated item obtains following equation:
It is acquired using least square methodSolution are as follows:
92) in order to improve memory efficient, the present invention is further introduced into intermediate variable Pk=RkCk,It is efficiently solved with incremental modeEquation is as follows:
Wherein intermediate variable Pk、XkAnd ZkUpdate rule are as follows:
Each initial guess is respectively as follows: P0=0, X0=0, Z0=0.
10) repeat the above steps 8), 9) until algorithmic statement, the at this moment result of iterationExactly preceding k frame image background Base Lk；
6) and 10) 11) B is solved: by the C respectively obtainedkAnd LkIt acquiresBkIt is exactly the last solution B of former problem.
12) each column of B are successively adjusted to the image of 115 × 86 pixels, finally obtain the back of the video sequence of recovery Scape (as shown in Figure 2).106 × 86 hallmonitor video frame is reused to input as initial data, other parameters are constant, Obtain the background image (as shown in Figure 3) of its recovery.

Claims (2)

1. a kind of video frequency sequence background restoration methods based on online low-rank background modeling, characterized in that steps are as follows:
1) background recovery problem in video sequence is specifically expressed as solving following unconstrained optimization equation:
Wherein | | | |FThis black norm of the not Luo Beini of representing matrix, | | | |*The nuclear norm of representing matrix, | | | |1It indicates One norm of matrix,Indicate that the point multiplication operation of two matrixes, D are that actual video sequence frame is successively arranged by column vector Matrix, B indicates background to be restored, and E represents foreground part, and W indicates that two-value sports ground maps weight matrix, λ1, λ2Table respectively Show the weight coefficient of low-rank background and prospect in video；
To solve nuclear norm needs singular value decomposition SVD is carried out to all frames together, low-rank matrix point is carried out to background B to be restored Solution:
Wherein inf { } expression takes infimum to { }, and L is the base of the corresponding lower-dimensional subspace of video frequency sequence background, and C is video Sequence corresponds to the coefficient matrix of base L；Obtain following decomposition model:
When solving this equation, using alternate optimization method, following loss function is minimized with incremental mode:
Wherein n indicates the current totalframes of video sequence, dkIndicate the column vector of video sequence kth frame,For kth frame figure As dkThe corresponding loss function at base L, is defined as follows:
Wherein | | | |2Indicate two norms of vector, w indicates present frame dkMovement map weight vectors, c is dkAt base L Coefficient vector, e are the foreground part of present frame；
Solution procedure is, online to each frame image dkSuccessively: estimation motion information calculates kth frame movement mapping weight vectors wk, update kth frame coefficient vector ck, update kth frame prospect ek, the corresponding base L of k frame before updatingk；
2) building kth frame movement mapping weight vectors wk: w is solved using backward motion estimation scheme onlinek；
It is based on sparse prospect it is assumed that two-value movement weight is all assigned a value of 1 to first frame；For subsequent frame, respectively with One frame is as reference frame, is then movement mapping by its binaryzation using the dense sports ground of optical flow algorithm estimation present frame；
3) equation (5) following sequence is converted into using alternating direction method ADM to solve:
In above formulaIndicate the value of variable c when being minimized objective function,Expression makes target letter The value of variable e when number is minimized, l is the number of iterations；Then method according to step 4), 5) is iterated solution and is Number vector ckWith prospect ek；
4) it solvesClosed solutions by solving least square problem acquire coefficient
Remove and solves kth frame coefficient vector c in formula (7)kObjective function in ckUnrelated item obtains following equation:
It is acquired using least square methodSolution are as follows:
WhereinIndicate movement mapping wkDiagonal matrix, i.e., by vector wkEach element be successively put into matrixMaster couple On linea angulata, I indicates unit matrix；
5) it solvesKth frame prospect is acquired using contraction operator
Remove and solves kth frame prospect e in formula (7)kObjective function in ekUnrelated item obtains following equation:
Formula (10) is solved using contraction operator:
Wherein eik, sik, wikRespectively indicate ek, sk, wkIth pixel value,
6) repeat the above steps 4), 5) untilWithThe threshold value of setting is converged to, at this moment the result of iterationWithIt is exactly Present frame dkCoefficient vector ckWith prospect ek, byThe coefficient matrix C of k frame image before obtainingk；
7) equation (3) following unconstrained optimization problem is converted to using substitution of variable to solve:
WhereinFor the variable newly introduced, Rk=[r1,...,rk] indicate the present frame that has acquired and before Video frame background, rk=dk-ekIndicate the background of the kth frame image acquired, WkTo map weight by the movement of preceding k frame The two-value sports ground that vector is constituted maps weight matrix, λ3For weight coefficient；
Equation (12) following sequence is converted into using alternating direction method ADM to solve:
Then method according to step 8), 9) is iterated solution and obtains variable YkWith the base L of video sequence lower-dimensional subspacek；
8) it solvesMode acquires pixel-by-pixel
Remove and solves Y in formula (13)kObjective function in YkUnrelated item obtains following equation:
Using least square method, calculates solve pixel-by-pixel:
WhereinIndicate base LkThe i-th row, yikIt indicatesThe element value of i-th row kth column, rikIndicate rkIth pixel value；
9) it solvesClosed solutions by solving least square problem acquire base
Remove and solves L in formula (13)kObjective function in LkUnrelated item obtains following equation:
It is acquired using least square methodSolution are as follows:
Further, intermediate variable P is introducedk=RkCk,It is efficiently solved with incremental mode
Wherein:
10) repeat the above steps 8), 9) untilWithThe threshold value of setting is converged to, at this moment the result of iterationIt is exactly preceding k The base L of frame image backgroundk；
6) and 10) 11) video background B is solved: by the C respectively obtainedkAnd LkIt acquiresBkIt is exactly the final of former problem Solve B.
2. the video frequency sequence background restoration methods as described in claim 1 based on online low-rank background modeling, characterized in that step Rapid 2) specific formula is:
Wherein wikIndicate that the movement of kth frame image ith pixel maps weight, as wkIth pixel value,WithPoint Not Biao Shi kth frame image ith pixel horizontal motion components and vertical motion component, τ is threshold for binaryzation sports ground Value.
CN201610526248.0A 2016-07-06 2016-07-06 Video frequency sequence background restoration methods based on online low-rank background modeling Expired - Fee Related CN106204477B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610526248.0A CN106204477B (en) 2016-07-06 2016-07-06 Video frequency sequence background restoration methods based on online low-rank background modeling

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610526248.0A CN106204477B (en) 2016-07-06 2016-07-06 Video frequency sequence background restoration methods based on online low-rank background modeling

Publications (2)

Publication Number Publication Date
CN106204477A CN106204477A (en) 2016-12-07
CN106204477B true CN106204477B (en) 2019-05-31

Family

ID=57466399

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610526248.0A Expired - Fee Related CN106204477B (en) 2016-07-06 2016-07-06 Video frequency sequence background restoration methods based on online low-rank background modeling

Country Status (1)

Country Link
CN (1) CN106204477B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106815858B (en) * 2016-12-30 2020-01-03 首都师范大学 Moving target extraction method and device
CN106815854B (en) * 2016-12-30 2020-05-15 西安交通大学 On-line video foreground and background separation method based on regular error modeling
CN109949252B (en) * 2019-04-15 2020-12-25 北京理工大学 Infrared image light spot removing method based on compensation coefficient fitting
CN111626942A (en) * 2020-03-06 2020-09-04 天津大学 Method for recovering dynamic video background based on space-time joint matrix
CN111798485B (en) * 2020-06-30 2022-07-19 武汉大学 Event camera optical flow estimation method and system enhanced by IMU
CN113313179B (en) * 2021-06-04 2024-05-31 西北工业大学 Noise image classification method based on l2p norm robust least square method

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102881002A (en) * 2012-07-11 2013-01-16 天津大学 Video background recovery method based on movement information and matrix completion
CN104867119A (en) * 2015-05-21 2015-08-26 天津大学 Structural lack image filling method based on low rank matrix reconstruction

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8935308B2 (en) * 2012-01-20 2015-01-13 Mitsubishi Electric Research Laboratories, Inc. Method for recovering low-rank matrices and subspaces from data in high-dimensional matrices

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102881002A (en) * 2012-07-11 2013-01-16 天津大学 Video background recovery method based on movement information and matrix completion
CN104867119A (en) * 2015-05-21 2015-08-26 天津大学 Structural lack image filling method based on low rank matrix reconstruction

Also Published As

Publication number Publication date
CN106204477A (en) 2016-12-07

Similar Documents

Publication Publication Date Title
CN106204477B (en) Video frequency sequence background restoration methods based on online low-rank background modeling
CN111814719B (en) Skeleton behavior recognition method based on 3D space-time diagram convolution
CN110287819B (en) Moving target detection method based on low rank and sparse decomposition under dynamic background
CN112434655B (en) Gait recognition method based on adaptive confidence map convolution network
CN105243670B (en) A kind of sparse and accurate extracting method of video foreground object of low-rank Combined expression
CN110570481A (en) calligraphy word stock automatic repairing method and system based on style migration
CN105931264B (en) A kind of sea infrared small target detection method
CN107229920B (en) Behavior identification method based on integration depth typical time warping and related correction
CN106408530A (en) Sparse and low-rank matrix approximation-based hyperspectral image restoration method
CN110827312B (en) Learning method based on cooperative visual attention neural network
CN110675422B (en) Video foreground and background separation method based on generalized non-convex robust principal component analysis
CN110139046A (en) A kind of video frame synthetic method based on tensor
CN106780450A (en) A kind of image significance detection method based on low-rank Multiscale Fusion
CN106815854A (en) A kind of Online Video prospect background separation method based on normal law error modeling
CN108924148A (en) A kind of source signal collaborative compression perception data restoration methods
CN106897971B (en) Non-local TV image denoising method based on independent component analysis and singular value decomposition
CN110826500A (en) Method for estimating 3D human body posture based on antagonistic network of motion link space
CN104376533A (en) Image denoising method based on regularization principal component tracking
CN109658362B (en) Data recovery method based on supplied core norm
CN104820967A (en) On-orbit calculation imaging method
Sagonas et al. Recovering joint and individual components in facial data
CN106296610A (en) The three-dimensional framework restorative procedure analyzed based on low-rank matrix
CN105631471A (en) Aurora sequence classification method with fusion of single frame feature and dynamic texture model
CN117611428A (en) Fashion character image style conversion method
CN105427351B (en) Compression of hyperspectral images cognitive method based on manifold structure sparse prior

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20190531