CN106296610A - The three-dimensional framework restorative procedure analyzed based on low-rank matrix - Google Patents

Abstract

The invention belongs to computer utility, three-dimensional framework recovery technique, for preferably recovering the movable information of three-dimensional body from damaged skeleton, minimize time cost simultaneously, the present invention adopts the technical scheme that, the three-dimensional framework restorative procedure analyzed based on low-rank matrix, utilize convex low-rank matrix Restoration model, by minimizing the sum of L1 norm and nuclear norm, correct the Error Elements in low-rank matrix, thus obtain a preferable matrix, thus repair the skeleton of breaking-up, existing accurate, the smooth reconstruction to compound movement.Present invention is mainly applied to three-dimensional framework and repair occasion.

Description

The three-dimensional framework restorative procedure analyzed based on low-rank matrix

Technical field

The invention belongs to computer application field, the reparation problem to three-dimensional framework.Invention describes one convex low Order matrix Restoration model, it is proposed that a kind of new three-dimensional motion restoration method analyzed based on low-rank matrix, it is possible to correct and extensive Multiple the most irrational and by the movable information of serious damage.

Background technology

Human motion is studied always computer graphics and the hot issue of computer vision field.The most therefore spread out Having given birth to a lot of research directions such as: three-dimensional motion is rebuild, human body attitude is estimated, three-dimensional body is motion-captured, and skeleton is followed the trail of, Three-dimensional model deformation etc..

Three-dimensional framework reparation is a major issue of three-dimensional body field of motion capture, in computer graphics and calculating Machine visual field suffers from extensive and practical important application.Traditional motion capture system is due to cost height, operating difficulties etc. Defect, is difficult to promote the use of always.In recent years, the depth camera with Kinect as representative can gather three-dimensional article conveniently and efficiently The movable information of body, has obtained quite varied application.But, especially have, for some complexity, the motion blocked, Kinect can not sufficiently accurately reconstruct movable information, and the skeleton collected has to pass through reparation and just can use.This is just Us are made to need an algorithm that can reconstruct three-dimensional body, especially human motion well.

The reconstruction of three-dimensional body movable information, typically, needs to collect the exercise data of good three-dimensional body, Rebuild on the basis of this.But, due to the restriction of collecting device, even such as the emerging popular camera of Kinect etc., the most very Difficulty collects the most errorless movable information.This process being accomplished by a lot of later stage and Optimization Work.There is a lot of existing algorithm The motion of three-dimensional body can be estimated according to RGB (colored) image or depth image.(K.Li, J.Yang, the and such as Li J.Jiang,“Nonrigid structure from motion viasparse representation.”IEEE Transactions on Cybernetics, vol.45, no.8, pp.1401 1413,2015.) propose with rarefaction representation Method estimates 3 d pose and camera position.(A.Toshev and the C.Szegedy, " Deeppose:Human such as Toshev pose estimation via deep neural networks,”in Proc.IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp.1653 1660.) propose the frame with degree of depth study The method of skeleton estimated by frame.The framework information of artificial destruction is put in deep neural network by they, allows network self-teaching Skeleton feature, tests with the skeleton damaged the most again.Training in advance, this method is needed yet with deep neural network While obtaining very well results the most quite time-consuming.(X.Wei, P.Zhang, and the J.Chai, " Accurate such as Wei realtime full-body motion capture using a single depth camera,”ACM Transactions on Graphics, vol.31, no.6, pp.439 445,2012.) incorporated by a depth camera The information such as depth data, human geometry's data, establish an automatic motion capture system, can catch and reconstruct human body Corresponding sports.But, this system, for repairing and having the skeleton blocked, there is also the space much had much room for improvement.As From damaged skeleton, how to recover the movable information of three-dimensional body, remain a challenging problem.

Summary of the invention

Owing to the motion of human skeleton has the highest temporal correlation, therefore, three-dimensional motion manifold should be present in one Among the subspace of low dimensional.When this is it is to say, be incorporated into framework information in a matrix, this matrix should be low-rank square Battle array.

In order to preferably recover the movable information of three-dimensional body from damaged skeleton, minimize time cost simultaneously, The present invention adopts the technical scheme that, the three-dimensional framework restorative procedure analyzed based on low-rank matrix, utilizes convex low-rank matrix to recover Model, by minimizing the sum of L1 norm and nuclear norm, corrects the Error Elements in low-rank matrix, thus obtains an ideal Matrix, thus repair the skeleton of breaking-up, it is achieved accurate, the smooth reconstruction to compound movement.

Described convex low-rank matrix is that damaged framework information is integrated low-rank matrix D obtained, low-rank matrix D every String, represents 21 nodes of skeleton respectively；The row of low-rank matrix D represents the three-dimensional global coordinate position of each frame skeleton node； Then matrix is carried out SVD decomposition and carries out low-rank checking.

Utilize convex low-rank matrix Restoration model, by minimizing the sum of L1 norm and nuclear norm, obtain a preferable square Battle array, comprises the concrete steps that, problem of being repaired by skeleton models:

D=A+E (1)

Wherein, D is that damaged framework information integrates the matrix constituted, and A is repairing of obtaining after matrix reparation Skeleton three-dimensional coordinate constitute matrix, E is matrix of errors；

min rank(A)+γ‖E‖₀S.t.D=A+E (2)

Wherein, rank (A) is the order of matrix A, ‖ E ‖₀Being the L-0 norm of matrix E, γ is the ratio between balance A and E Weight weight term, γ ＞ 0, owing to above-mentioned equation is NP-double linear problems of difficulty for solving, so above-mentioned equation is redescribed into,

min ‖A‖_*+λ‖E‖₁S.t.D=A+E (3)

Wherein, ‖ A ‖_*It is the nuclear norm of matrix A,σ_iIt is the singular value of matrix A, ‖ E‖₁Being the L1 norm of matrix E, λ ＞ 0, is a weight coefficient；Augmented Lagrange method is utilized finally to solve.

Utilize Augmented Lagrange method finally to solve to comprise the concrete steps that, introduce and reduce variable and thresholding variable, and Solve the linear equation of low-rank matrix, then solve convex optimization method respectively:

The Lagrange's equation of equation (3) is:

$L(A,E,Y,\mathrm{\&mu;})=\left|\right|A|{|}_{*}+\mathrm{\&lambda;}|\left|E\right|{|}_1+<Y,D-A-E>+\frac{\mathrm{\&mu;}}{2}\left|\right|D-A-E|{|}_F^2---\left(4\right)$

Wherein, A-problem:E-problem: Y-problem:Y^k+1=Y^k+ μ (D-A^k+1-E^k+1), μ^k+1=ρμ^k, ρ ＞ 1；

Wherein | | | |_FRepresent is the F norm of matrix, the S in E-problem_δX () is to reduce variable, wherein S_δ(x)=sgn (x) max (| x |-δ, 0)；M in A-problem_δX () is singular value door Limit variable, whereinM_δ(x)=US_δ(Λ) V, U, V be respectively x is carried out singular value decomposition after Left and right eigenvectors matrix, Λ is diagonal matrix, and the element on diagonal is the singular value of x；λ and μ is positive constant, Y It is Lagrange multiplier, ＜, > represent the inner product that two matrixes are regarded as long vector；

Solve convex optimization method the most respectively:

${\mathrm{argmin}}_{A}L(A,E,Y)=S_{\frac{\mathrm{\&lambda;}}{\mathrm{\&mu;}}}(D-A+\frac{1}{\mathrm{\&mu;}}Y)---\left(5\right)$

${\mathrm{argmin}}_{E}L(A,E,Y)=M_{\frac{1}{\mathrm{\&mu;}}}(D-E+\frac{1}{\mathrm{\&mu;}}Y)---\left(6\right)$

Under the framework of augmentation Lagrange solution, λ, μ and Y can effectively update, and variable E, A are iterated minimum Change, update Lagrange multiplier Y, finally give reparation matrix A.

The feature of the present invention and providing the benefit that:

The algorithm that present invention low-rank matrix is recovered has repaired the skeleton motion information damaged, and completes bone on this basis The three-dimensional reconstruction target of frame.It has the following characteristics that

1, being easily understood, complexity is relatively low, it is easy to accomplish.

2, the Method Modeling that low-rank matrix is recovered is utilized, it is achieved the time is relatively short, respond well.

3, rank of matrix is difficult to definition, therefore substitutes with matrix nuclear norm.Restrictive best L0 norm has non- Convexity, this makes to solve and becomes extremely difficult.So the optimum convex approximation L1 norm that we use L0 norm retrains, L1 model It is convex optimization problem that number minimizes, and can carry out solving of linear equation.

4, the linear equation of low-rank matrix is solved by the method for augmentation Lagrange.

Accompanying drawing illustrates:

The present invention above-mentioned and/or that add aspect and advantage will become from the following description of the accompanying drawings of embodiments Substantially with easy to understand:

Fig. 1 is the method flow diagram of the inventive method；

Fig. 2 is that application low-rank matrix analyzes the skeleton comparison diagram after method is repaired；

Fig. 2 (a) is the cromogram that Kinect gathers；

Fig. 2 (b) is the skeleton display figure that Kinect gathers；

Fig. 2 (c) is the skeleton display figure after low-rank matrix recovery processes.

Detailed description of the invention

The present invention utilizes the method that low-rank matrix is analyzed, and repairs the breaking-up framework information collected, thus realizes The reconstruction of three-dimensional motion information.Please audit and see and could send in one's application by such text.

In order to preferably recover the movable information of three-dimensional body from damaged skeleton, minimize time cost simultaneously, The present invention adopts the technical scheme that, utilizes convex low-rank matrix Restoration model, by minimizing the sum of L1 norm and nuclear norm, comes Correct the Error Elements in low-rank matrix, thus obtain a preferable matrix, thus repair the skeleton damaged, existing to complexity Accurate, the smooth reconstruction of motion.Concrete grammar comprises the following steps:

1) damaged framework information is incorporated in matrix D.Wherein, every string of D, show respectively 21 joints of skeleton Point, the row of D, represent the three-dimensional global coordinate position of each frame skeleton node.

2) validation matrix low-rank:

Matrix is carried out SVD decomposition and carries out low-rank checking.

3) problem of being repaired by skeleton models:

D=A+E (1)

Wherein, D is the matrix that the skeleton three-dimensional coordinate information damaged is constituted, and A is repairing of obtaining after matrix reparation The matrix that the skeleton three-dimensional coordinate answered is constituted, E is matrix of errors.According to the temporal correlation of skeleton motion information, matrix A is also It should be low-rank.

min rank(A)+γ‖E‖₀S.t.D=A+E (2)

Wherein, rank (A) is the order of matrix A, ‖ E ‖₀Being the L-0 norm of matrix E, γ is the ratio between balance A and E The weight term of weight, γ ＞ 0.Owing to above-mentioned equation is NP-double linear problems of difficulty for solving, thus above-mentioned equation is redescribed into,

min ‖A‖_*+λ‖E‖₁S.t.D=A+E (3)

Wherein, ‖ A ‖_*It is the nuclear norm of matrix A,σ_iIt it is the singular value of matrix A.‖ E‖₁Being the L-1 norm of matrix E, λ ＞ 0, is a weight coefficient.The purpose of do so is, nuclear norm can preferably substitute square The order of battle array A, for the L-0 norm of matrix E, the L-1 norm of matrix E is convex function, and equation solution process is more convenient.

4) Augmented Lagrange method is utilized finally to solve

Utilize Augmented Lagrange method finally to solve to comprise the concrete steps that, introduce and reduce variable and thresholding variable, and Solve the linear equation of low-rank matrix, then solve convex optimization method respectively.

The Lagrange's equation of equation (3) is:

$L(A,E,Y,\mathrm{\&mu;})=\left|\right|A|{|}_{*}+\mathrm{\&lambda;}|\left|E\right|{|}_1+<Y,D-A-E>+\frac{\mathrm{\&mu;}}{2}\left|\right|D-A-E|{|}_F^2---\left(4\right)$

Wherein, A-problem:E-problem: Y-problem:Y^k+1=Y^k+μ(D-A^k+1-E^k+1), μ^k+1=ρ μ^k, ρ ＞ 1.

Wherein | | | |_FRepresent is the F norm of matrix, the S in E-problem_δX () is to reduce variable, whereinS_δ(x)=sgn (x) max (| x |-δ, 0)；M in A-problem_δX () is singular value door Limit variable, whereinM_δ(x)=US_δ(Λ) V, U, V be respectively x is carried out singular value decomposition after Left and right eigenvectors matrix, Λ is diagonal matrix, and the element on diagonal is the singular value of x；λ and μ is positive constant, Y Being Lagrange multiplier,<,>represents the inner product that two matrixes are regarded as long vector；

Solve convex optimization method the most respectively:

${\mathrm{argmin}}_{A}L(A,E,Y)=S_{\frac{\mathrm{\&lambda;}}{\mathrm{\&mu;}}}(D-A+\frac{1}{\mathrm{\&mu;}}Y)---\left(5\right)$

${\mathrm{argmin}}_{E}L(A,E,Y)=M_{\frac{1}{\mathrm{\&mu;}}}(D-E+\frac{1}{\mathrm{\&mu;}}Y)---\left(6\right)$

Under the framework of augmentation Lagrange solution, λ, μ and Y can effectively update, and variable E, A are iterated minimum Change, update Lagrange multiplier Y, finally give reparation matrix A.

The present invention is further described below in conjunction with the accompanying drawings with detailed description of the invention.

The present invention utilizes convex low-rank matrix Restoration model, by minimizing the sum of L1 norm and nuclear norm, corrects low-rank Error Elements in matrix, thus repaired the skeleton damaged, existing accurate, the smooth reconstruction to compound movement.In the accompanying drawings It can be seen that after algorithm process, the skeleton of former breaking-up has obtained good reparation.

1) damaged framework information is incorporated in matrix.Wherein, every string of D, show respectively 21 joints of skeleton Point, the row of D, represent the three-dimensional global coordinate position of each frame skeleton node.

2) validation matrix low-rank:

Owing to the motion of human skeleton has the highest temporal correlation, therefore, the skeleton matrix D of integration should be low-rank square Battle array.Matrix is carried out SVD decomposition and carries out low-rank checking.

3) problem of being repaired by skeleton models:

D=A+E (1)

Wherein, D is the matrix that the skeleton three-dimensional coordinate information damaged is constituted, and A is repairing of obtaining after matrix reparation The matrix that the skeleton three-dimensional coordinate answered is constituted, E is matrix of errors.According to the temporal correlation of skeleton motion information, matrix A is also It should be low-rank.

min rank(A)+γ‖E‖₀S.t.D=A+E (2)

Wherein, rank (A) is the order of matrix A, ‖ E ‖₀Being the L-0 norm of matrix E, γ is the ratio between balance A and E The weight term of weight, γ ＞ 0.Owing to above-mentioned equation is NP-double linear problems of difficulty for solving, thus above-mentioned equation is redescribed into,

min ‖A‖_*+λ‖E‖₁S.t.D=A+E (3)

Wherein, ‖ A ‖_*It is the nuclear norm of matrix A,σ_iIt it is the singular value of matrix A.‖ E‖₁Being the L-1 norm of matrix E, λ ＞ 0, is a weight coefficient.The purpose of do so is, nuclear norm can preferably substitute square The order of battle array A, for the L-0 norm of matrix E, the L-1 norm of matrix E is convex function, and equation solution process is more convenient.

4) Augmented Lagrange method is utilized finally to solve

Utilize Augmented Lagrange method finally to solve to comprise the concrete steps that, introduce and reduce variable and thresholding variable, and Solve the linear equation of low-rank matrix, then solve convex optimization method respectively.

The Lagrange's equation of equation (3) is:

$L(A,E,Y,\mathrm{\&mu;})=\left|\right|A|{|}_{*}+\mathrm{\&lambda;}|\left|E\right|{|}_1+<Y,D-A-E>+\frac{\mathrm{\&mu;}}{2}\left|\right|D-A-E|{|}_F^2---\left(4\right)$

Wherein, A-problem:E-problem: Y-problem:Y^k+1=Y^k+μ(D-A^k+1-E^k+1), μ^k+1=ρ μ^k, ρ ＞ 1.

Wherein | | | |_FRepresent is the F norm of matrix, the S in E-problem_δX () is to reduce variable, whereinS_δ(x)=sgn (x) max (| x |-δ, O)；M in A-problem_δX () is singular value door Limit variable, whereinM_δ(x)=US_δ(Λ) V, U, V be respectively x is carried out singular value decomposition after Left and right eigenvectors matrix, Λ is diagonal matrix, and the element on diagonal is the singular value of x；λ and μ is positive constant, Y Being Lagrange multiplier, ＜, ＞ represent the inner product that two matrixes are regarded as long vector；

Solve convex optimization method the most respectively:

${\mathrm{argmin}}_{A}L(A,E,Y)=S_{\frac{\mathrm{\&lambda;}}{\mathrm{\&mu;}}}(D-A+\frac{1}{\mathrm{\&mu;}}Y)---\left(5\right)$

${\mathrm{argmin}}_{E}L(A,E,Y)=M_{\frac{1}{\mathrm{\&mu;}}}(D-E+\frac{1}{\mathrm{\&mu;}}Y)---\left(6\right)$

Under the framework of augmentation Lagrange solution, λ, μ and Y can effectively update, and variable E, A are iterated minimum Change, update Lagrange multiplier Y, finally give reparation matrix A.

Claims (4)

1. the three-dimensional framework restorative procedure analyzed based on low-rank matrix, is characterized in that, utilize convex low-rank matrix Restoration model, By minimizing the sum of L1 norm and nuclear norm, correct the Error Elements in low-rank matrix, thus obtain a preferable square Battle array, thus repair the skeleton of breaking-up, it is achieved accurate, the smooth reconstruction to compound movement.

2. the three-dimensional framework restorative procedure analyzed based on low-rank matrix as claimed in claim 1, is characterized in that, described convex low-rank Matrix is that damaged framework information is integrated low-rank matrix D obtained, every string of low-rank matrix D, represents skeleton respectively 21 nodes；The row of low-rank matrix D represents the three-dimensional global coordinate position of each frame skeleton node；Then matrix is carried out SVD Decompose and carry out low-rank checking.

3. the three-dimensional framework restorative procedure analyzed based on low-rank matrix as claimed in claim 1, is characterized in that, utilize convex low-rank Matrix Restoration model, by minimizing the sum of L1 norm and nuclear norm, obtains a preferable matrix, thus repairs the bone of breaking-up Frame comprises the concrete steps that,

1) problem of being repaired by skeleton models:

D=A+E (1)

Wherein, D is that damaged framework information integrates the matrix constituted, and A is the bone repaired obtained after matrix reparation The matrix that frame three-dimensional coordinate is constituted, E is matrix of errors；

min rank(A)+γ‖E‖₀S.t.D=A+E (2)

Wherein, rank (A) is the order of matrix A, ‖ E ‖₀Being the L-0 norm of matrix E, γ is the proportion between balance A and E Weight term, γ ＞ 0, owing to above-mentioned equation is NP-double linear problems of difficulty for solving, thus above-mentioned equation is redescribed into,

min‖A‖_*+λ‖E‖₁S.t.D=A+E (3)

Wherein, ‖ A ‖_*It is the nuclear norm of matrix A,σ_iIt is the singular value of matrix A, ‖ E ‖₁It is The L-1 norm of matrix E, λ ＞ 0, is a weight coefficient；Augmented Lagrange method is utilized finally to solve.

4. the three-dimensional framework restorative procedure analyzed based on low-rank matrix as claimed in claim 3, is characterized in that, utilize augmentation to draw Ge Lang method finally solves and comprises the concrete steps that, introduces and reduces variable and thresholding variable, and solves the linear of low-rank matrix Equation, then solve convex optimization method respectively:

The Lagrange's equation of equation (3) is:

$L(A,E,Y,\mathrm{\&mu;})=\left|\right|A|{|}_{*}+\mathrm{\&lambda;}|\left|E\right|{|}_1+<Y,D-A-E>+\frac{\mathrm{\&mu;}}{2}\left|\right|D-A-E|{|}_F^2---\left(4\right)$

Wherein, Y-problem:Y^k+1=Y^k+μ(D-A^k+1-E^k+1), μ^k+1=ρ μ^k, ρ ＞ 1；

Wherein | | | |_FRepresent is the F norm of matrix, the S in E-problem_δX () is to reduce variable, whereinS_δ(x)=sgn (x) max (| x |-δ, 0)；M in A-problem_δX () is singular value door Limit variable, whereinM_δ(x)=US_δ(Λ) V, U, V be respectively x is carried out singular value decomposition after Left and right eigenvectors matrix, Λ is diagonal matrix, and the element on diagonal is the singular value of x；λ and μ is positive constant, Y Being Lagrange multiplier,<,>represents the inner product that two matrixes are regarded as long vector；

Solve convex optimization method the most respectively:

${\mathrm{argmin}}_{A}L(A,E,Y)=S_{\frac{\mathrm{\&lambda;}}{\mathrm{\&mu;}}}(D-A+\frac{1}{\mathrm{\&mu;}}Y)---\left(5\right)$

${\mathrm{argmin}}_{E}L(A,E,Y)=M_{\frac{1}{\mathrm{\&mu;}}}(D-E+\frac{1}{\mathrm{\&mu;}}Y)---\left(6\right)$

Under the framework of augmentation Lagrange solution, λ, μ and Y can effectively update, and are iterated minimizing to variable E, A, more New Lagrange multiplier Y, finally gives reparation matrix A.