CN108805974A - A kind of intensive non-rigid motion structure algorithm based on Grassmann manifold - Google Patents

A kind of intensive non-rigid motion structure algorithm based on Grassmann manifold Download PDF

Info

Publication number
CN108805974A
CN108805974A CN201810520065.7A CN201810520065A CN108805974A CN 108805974 A CN108805974 A CN 108805974A CN 201810520065 A CN201810520065 A CN 201810520065A CN 108805974 A CN108805974 A CN 108805974A
Authority
CN
China
Prior art keywords
grassmann
manifold
space
analysis
formulation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN201810520065.7A
Other languages
Chinese (zh)
Inventor
夏春秋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Vision Technology Co Ltd
Original Assignee
Shenzhen Vision Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Vision Technology Co Ltd filed Critical Shenzhen Vision Technology Co Ltd
Priority to CN201810520065.7A priority Critical patent/CN108805974A/en
Publication of CN108805974A publication Critical patent/CN108805974A/en
Withdrawn legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00Computer-aided design [CAD]

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Software Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

A kind of intensive non-rigid motion structure algorithm based on Grassmann manifold proposed in the present invention, its main contents includes Grassmann manifold, non-rigid motion structure formulation and three-dimensional reconstruction experiment and analysis, its process is that carrying out Jim Glassman formulation to trajectory range first indicates;Then, Jim Glassman formulation is carried out to shape space to indicate;Next, carrying out space-time formulation and carrying out bilinearity optimization;Finally, it carries out three-dimensional reconstruction experiment and carries out control variable analysis and motion time analysis.The present invention is based on Grassmann manifolds not to need any Prior Template when handling non-rigid motion structure, can handle with noisy nonlinear deformation and be capable of providing the final result based on basic data collection;For the present invention with better data scalability and with higher reconstruct accuracy, comprehensive performance is optimal.

Description

Grassmann manifold-based dense non-rigid motion structure algorithm
Technical Field
The invention relates to the field of three-dimensional reconstruction, in particular to a dense non-rigid motion structure algorithm based on Grassmann manifold.
Background
The three-dimensional reconstruction technology describes a real scene into a mathematical model conforming to the logical expression of a computer through the processes of depth data acquisition, preprocessing, point cloud registration and fusion, surface generation and the like, and relates to a plurality of subject systems including image processing, stereoscopic vision, mode identification and the like. In clinical medicine, a three-dimensional reconstruction technology can present a two-dimensional organ image in a three-dimensional form, and is beneficial to analysis and planning before an operation; in the cultural relic exhibition field, the three-dimensional reconstruction technology can be used for digitally processing precious cultural relics to build a network-based digital museum system, and can better solve the contradiction between rich exhibits and limited space and time; in addition, the three-dimensional reconstruction technology has important application in the fields of game development, industrial design, aerospace, navigation and the like. However, the existing three-dimensional reconstruction algorithm has the problems of low data expandability, low reconstruction accuracy and the like.
According to the dense non-rigid motion structure algorithm based on the Grassman manifold, Grassman formula expression is firstly carried out on a track space; then, performing Grassmann formulation expression on the shape space; then, performing space-time formulation and bilinear optimization; and finally, carrying out a three-dimensional reconstruction experiment and carrying out control variable analysis and runtime analysis. The invention does not need any prior template when processing the non-rigid motion structure based on the Grassmann manifold, can process the nonlinear deformation with noise and can provide the final result based on the basic data set; the invention has better data expandability, higher reconstruction accuracy and optimal comprehensive performance.
Disclosure of Invention
Aiming at the problems of low data expandability, low reconstruction accuracy and the like of the existing three-dimensional reconstruction algorithm, the invention aims to provide a dense non-rigid motion structure algorithm based on the Grassman manifold, which firstly carries out Grassman formula expression on a track space; then, performing Grassmann formulation expression on the shape space; then, performing space-time formulation and bilinear optimization; and finally, carrying out a three-dimensional reconstruction experiment and carrying out control variable analysis and runtime analysis.
In order to solve the above problems, the present invention provides a dense non-rigid motion structure algorithm based on grassmann manifold, which mainly comprises:
(one) Grassmann manifold;
(II) formulating a non-rigid motion structure;
and (III) three-dimensional reconstruction experiment and analysis.
Wherein said Grassmann manifold, generally denoted asByAll r dimensions of (1) are linear subspace composition, n is more than r; each point on the Grassmann manifold is represented by an n X r matrix (called X), where each column is made up of X tiles, denoted span (X) or abbreviated [ X ]]。
Wherein, the non-rigid motion structure formulation mainly comprises: a grassmann representation of trajectory space, a grassmann representation of shape space, a spatio-temporal formulation and a formulation operation.
Further, the grassmann manifold representation of the trajectory space is isomerously equivalent to a symmetric idempotent matrix, the grassmann manifold can be embedded in a symmetric matrix manifold, and self-expression can be defined in the embedding space:
wherein,representing a tensor constructed by mapping a symmetric matrix of all grassmann data points;representing a matrix of coefficients (K)sTotal number of spatial groups); esAnd performing track group reconstruction error measurement on each manifold geometry.
Further, the grassmann representation of the shape space assumes that temporal deformation proceeds smoothly over time, and the deformed shape is based on local self-expression across frames:
whereinRepresenting a tensor constructed by mapping a symmetric matrix of all grassmann data points;representing a matrix of coefficients (K)tTotal number of time groups); etTime group reconstruction error measurements are made for each manifold geometry.
Further, the space-time formulation combines the grassmann representation of the trajectory space and the grassmann representation of the shape space, and a combination formula (comprising the whole of the local space structure and the non-rigid shape) can be obtained by the constraint conditions and the reprojection error terms of the two:
wherein,Ψs=ξ(Cs,Ss,σ),Sc=ζ(Ψs,∑s,VS,NS),w for reprojection error constraintsAnd R represents a three-dimensional reconstruction.
Further, the formula operates becauseIs a bilinear optimization variable, so the formula (3) is a non-convex problem and is difficult to solve; however, the non-convex problem can be efficiently solved using Augmented Lagrange Methods (ALMs); multiplying lagrange multiplierAnd arbitrary variables (J)s,Jt) Substituting equation (3) with:
therein, Ψs=ξ(Cs,Ss,σ),Ss=ζ(Ψs,∑s,VS,NS), first, the formula ξ (·) calculates the Singular Value Decomposition (SVD) of the C matrix, i.e., C ═ U ·c,∑c,Vc]Then form a matrix A, i.e.Wherein sigmaIs based on the noise level and X ═ Uc(∑c)0.5Obtaining an empirical value; secondly, with AijThe grassmann samples are constructed from the S matrix, which reduces the task complexity from vector point index to linear subspace index.
Wherein, the three-dimensional reconstruction experiment and analysis, for the experiment, in order to carry out the quantitative evaluation of the three-dimensional reconstruction, the estimated shape of each frame isAnd the actual shape of the labelArranging up; mean root mean square three-dimensional reconstruction error formulaInitializing the segments by adopting a Kmeans + + algorithm; for the analysis, mainly control variable analysis and runtime analysis are included.
The analysis of the control variables is characterized in that the analysis of the control variables is mainly used for evaluating the importance of space-time limitation in formulation; the performance of the main observation formula is shown in four cases: (a) no space-time limitation; (b) only time limitations exist; (c) only limited in space; (d) with time and space limitations.
The runtime analysis is characterized in that the method is mainly used for comparing with other dense non-rigid motion structure reconstruction methods; the result shows that the expandability of the method in the data set exceeds 50000-70000 points, the method exceeds most methods and has higher reconstruction accuracy, and the comprehensive performance is optimal.
Drawings
FIG. 1 is a system framework diagram of a Grassmann manifold-based dense non-rigid motion structure algorithm of the present invention.
FIG. 2 is a Grassmann representation of a Grassmann manifold-based dense non-rigid motion structure algorithm of the present invention.
FIG. 3 is a three-dimensional reconstruction effect diagram of an image deblurring method based on the total variation of the weighted graph.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application can be combined with each other without conflict, and the present invention is further described in detail with reference to the drawings and specific embodiments.
FIG. 1 is a system framework diagram of a Grassmann manifold-based dense non-rigid motion structure algorithm of the present invention. The method mainly comprises the following steps of Grassmann manifold, non-rigid motion structure formulation and three-dimensional reconstruction experiment and analysis.
Grassmann manifold, commonly denoted asByAll r dimensions of (1) are linear subspace composition, n is more than r; grid (C)Each point on the Raman manifold is represented by an n X r matrix (called X), where each column is made up of X tiles, denoted as span (X) or abbreviated as [ X ]]。
The non-rigid motion structure formulation mainly comprises the following steps: a grassmann representation of trajectory space, a grassmann representation of shape space, a spatio-temporal formulation and a formulation operation.
The space-time formulation is realized by combining a Grassmann representation method of a track space and a Grassmann representation method of a shape space, and a combined formula (comprising a local space structure and a non-rigid shape) can be obtained by constraint conditions of the two and a reprojection error term:
wherein,Ψs=ξ(Cs,Ss,σ),Ss=ζ(Ψs,∑s,VS,NS),the reprojection error constraint represents a three-dimensional reconstruction with Ws and R.
Wherein the formula operation is characterized in thatIs a bilinear optimization variable, so the formula (3) is a non-convex problem and is difficult to solve; however, the non-convex problem can be efficiently solved using Augmented Lagrange Methods (ALMs); multiplying lagrange multiplierAnd arbitrary variables (J)s,Jt) Substituting equation (3) with:
therein, Ψs=ξ(Cs,Ss,σ),Ss=ζ(Ψs,∑s,VS,Ns), first, the formula ξ (·) calculates the Singular Value Decomposition (SVD) of the C matrix, i.e., C ═ U ·c,∑c,Vc]Then form a matrix A, i.e.Wherein sigmaIs based on the noise level and X ═ Uc(∑c)0.5Obtaining an empirical value; second, the grassmann samples are constructed from the S matrix with Aij, which reduces the task complexity from vector point index to linear subspace index.
Three-dimensional reconstruction experiment and analysis, for the experiment, in order to carry out quantitative evaluation of three-dimensional reconstruction, the estimated shape of each frameAnd the actual shape of the labelArranging up; mean root mean square three-dimensional reconstruction error formula Initializing the segments by adopting a Kmeans + + algorithm; for the analysis, mainly control variable analysis and runtime analysis are included.
The analysis of the control variables is characterized in that the analysis of the control variables is mainly used for evaluating the importance of space-time limitation in formulation; the performance of the main observation formula is shown in four cases: (a) no space-time limitation; (b) only time limitations exist; (c) only limited in space; (d) with time and space limitations.
The runtime analysis is characterized in that the method is mainly used for comparing with other dense non-rigid motion structure reconstruction methods; the result shows that the expandability of the method in the data set exceeds 50000-70000 points, the method exceeds most methods and has higher reconstruction accuracy, and the comprehensive performance is optimal.
FIG. 2 is a Grassmann representation of a Grassmann manifold-based dense non-rigid kinematic structure algorithm. Including a grassmann representation of the trajectory space and a grassmann representation of the shape space.
The Grassmann representation method of the track space, the Grassmann manifold is equivalent to the symmetric power matrix isomerously, the Grassmann manifold can be embedded in the symmetric matrix manifold, and the self-expression can be defined in the embedding space:
wherein,representing a tensor constructed by mapping a symmetric matrix of all grassmann data points;representing a matrix of coefficients (K)sTotal number of spatial groups); esAnd performing track group reconstruction error measurement on each manifold geometry.
The grassmann representation of shape space is characterized by the fact that, assuming that the temporal deformation proceeds smoothly over time, the deformation shape is based on local self-expression across frames:
wherein,representing a tensor constructed by mapping a symmetric matrix of all grassmann data points;representing a matrix of coefficients (K)tTotal number of time groups); etTime group reconstruction error measurements are made for each manifold geometry.
FIG. 3 is a three-dimensional reconstruction effect diagram of an image deblurring method based on the total variation of the weighted graph. The three-dimensional reconstruction process comprises the steps of firstly carrying out Grassmann formulation expression on a track space; then, performing Grassmann formulation expression on the shape space; then, performing space-time formulation and bilinear optimization; and finally, carrying out a three-dimensional reconstruction experiment and carrying out control variable analysis and runtime analysis. The invention does not need any prior template when processing the non-rigid motion structure based on the Grassmann manifold, can process the nonlinear deformation with noise and can provide the final result based on the basic data set; the invention has better data expandability, higher reconstruction accuracy and optimal comprehensive performance.
It will be appreciated by persons skilled in the art that the invention is not limited to details of the foregoing embodiments and that the invention can be embodied in other specific forms without departing from the spirit or scope of the invention. In addition, various modifications and alterations of this invention may be made by those skilled in the art without departing from the spirit and scope of this invention, and such modifications and alterations should also be viewed as being within the scope of this invention. It is therefore intended that the following appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.

Claims (10)

1. A dense non-rigid motion structure algorithm based on a Grassmann manifold is characterized by mainly comprising a Grassmann manifold (I); formulating a non-rigid motion structure; and (3) three-dimensional reconstruction experiment and analysis.
2. Grassmann manifold (i) according to claim 1, wherein the grassmann manifold is generally expressed as grasmann manifoldByAll r dimensions of (1) are linear subspace composition, n is more than r; each point on the Grassmann manifold is represented by an n X r matrix (called X), where each column is made up of X tiles, denoted span (X) or abbreviated [ X ]]。
3. The non-rigid kinematic structure formulation (ii) according to claim 1, characterized by essentially comprising: a grassmann representation of trajectory space, a grassmann representation of shape space, a spatio-temporal formulation and a formulation operation.
4. The grassmann representation method of trajectory space according to claim 3, wherein the grassmann manifold is isomorphically equivalent to a symmetric idempotent matrix, the grassmann manifold can be embedded in a symmetric matrix manifold, and self-expression can be defined in the embedding space:
wherein,representing a tensor constructed by mapping a symmetric matrix of all grassmann data points;representing a matrix of coefficients (K)sTotal number of spatial groups); esAnd performing track group reconstruction error measurement on each manifold geometry.
5. Grassmann representation of shape space according to claim 3, wherein the shape deformation is based on local self-expression across frames, assuming that the temporal deformation proceeds smoothly over time:
wherein,representing a tensor constructed by mapping a symmetric matrix of all grassmann data points;representing a matrix of coefficients (K)tTotal number of time groups); etTime group reconstruction error measurements are made for each manifold geometry.
6. The spatio-temporal formulation according to claim 3, wherein the grassmann representation of the trajectory space and the grassmann representation of the shape space are combined, and a combination formula (comprising the local spatial structure and the non-rigid shape as a whole) is obtained by the constraint conditions of the two and the reprojection error term:
wherein,Ψs=ξ(Cs,Ss,σ),Ss=ζ(Ψs,∑s,VS,NS),w for reprojection error constraintsAnd R represents a three-dimensional reconstruction.
7. Operation on the basis of the formula according to claim 3, characterised in thatIs a bilinear optimization variable, so the formula (3) is a non-convex problem and is difficult to solve; however, the non-convex problem can be efficiently solved using Augmented Lagrange Methods (ALMs); multiplying lagrange multiplierAnd arbitrary variables (J)s,Jt) Substituting equation (3) with:
therein, Ψs=ξ(Cs,Ss,σ),Ss=ζ(Ψs,∑s,VS,NS), first, the formula ξ (·) calculates the Singular Value Decomposition (SVD) of the C matrix, i.e., C ═ U ·c,∑c,Vc]Then form a matrix A, i.e.Wherein sigmaIs based on the noise level and X ═ Uc(∑c)0.5Obtaining an empirical value; secondly, with AijConstructing the Grassmann sample from the S matrix reduces the task of indexing from vector points to linear subspace indexesComplexity.
8. Three-dimensional reconstruction experiment and analysis (iii) as claimed in claim 1, wherein, for the experiment, the estimated shape of each frame is evaluated for quantitative evaluation of the three-dimensional reconstructionAnd the actual shape of the labelArranging up; mean root mean square three-dimensional reconstruction error formulaInitializing the segments by adopting a Kmeans + + algorithm; for the analysis, mainly control variable analysis and runtime analysis are included.
9. The control variable analysis of claim 8, wherein the control variable analysis is primarily for the purpose of assessing the importance of spatio-temporal constraints in the formulation; the performance of the main observation formula is shown in four cases: (a) no space-time limitation; (b) only time limitations exist; (c) only limited in space; (d) with time and space limitations.
10. The runtime analysis of claim 8, mainly used for comparison with other dense non-rigid motion structure reconstruction methods; the result shows that the expandability of the method in the data set exceeds 50000-70000 points, the method exceeds most methods and has higher reconstruction accuracy, and the comprehensive performance is optimal.
CN201810520065.7A 2018-05-28 2018-05-28 A kind of intensive non-rigid motion structure algorithm based on Grassmann manifold Withdrawn CN108805974A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810520065.7A CN108805974A (en) 2018-05-28 2018-05-28 A kind of intensive non-rigid motion structure algorithm based on Grassmann manifold

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810520065.7A CN108805974A (en) 2018-05-28 2018-05-28 A kind of intensive non-rigid motion structure algorithm based on Grassmann manifold

Publications (1)

Publication Number Publication Date
CN108805974A true CN108805974A (en) 2018-11-13

Family

ID=64090486

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810520065.7A Withdrawn CN108805974A (en) 2018-05-28 2018-05-28 A kind of intensive non-rigid motion structure algorithm based on Grassmann manifold

Country Status (1)

Country Link
CN (1) CN108805974A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110007754A (en) * 2019-03-06 2019-07-12 清华大学 The real-time reconstruction method and device of hand and object interactive process
CN110414530A (en) * 2019-07-22 2019-11-05 上海交通大学 Image based on Riemann manifold optimization goes mixing impulse noise method and system
CN118484668A (en) * 2024-07-15 2024-08-13 山东济矿鲁能煤电股份有限公司阳城煤矿 Intelligent data processing method for intelligent coal face

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105488523A (en) * 2015-11-26 2016-04-13 沈阳大学 Data clustering analysis method based on Grassmann manifold
CN107609580A (en) * 2017-08-29 2018-01-19 天津大学 A kind of low-rank tensor identification analysis method of direct-push

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105488523A (en) * 2015-11-26 2016-04-13 沈阳大学 Data clustering analysis method based on Grassmann manifold
CN107609580A (en) * 2017-08-29 2018-01-19 天津大学 A kind of low-rank tensor identification analysis method of direct-push

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
SURYANSH KUMAR ET AL: ""Scalable Dense Non-rigid Structure-from-Motion: A Grassmannian Perspective"", 《ARXIV 1803.00233V2[CS.CV]》 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110007754A (en) * 2019-03-06 2019-07-12 清华大学 The real-time reconstruction method and device of hand and object interactive process
CN110007754B (en) * 2019-03-06 2020-08-28 清华大学 Real-time reconstruction method and device for hand-object interaction process
CN110414530A (en) * 2019-07-22 2019-11-05 上海交通大学 Image based on Riemann manifold optimization goes mixing impulse noise method and system
CN118484668A (en) * 2024-07-15 2024-08-13 山东济矿鲁能煤电股份有限公司阳城煤矿 Intelligent data processing method for intelligent coal face

Similar Documents

Publication Publication Date Title
Narita et al. Panopticfusion: Online volumetric semantic mapping at the level of stuff and things
Moon et al. Deephandmesh: A weakly-supervised deep encoder-decoder framework for high-fidelity hand mesh modeling
Wang et al. A computationally efficient semantic slam solution for dynamic scenes
Wang et al. IK-SVD: dictionary learning for spatial big data via incremental atom update
CN110458939A (en) The indoor scene modeling method generated based on visual angle
Castillo et al. Least median of squares filtering of locally optimal point matches for compressible flow image registration
CN107203988B (en) A kind of method and its application for rebuilding three-dimensional volumetric image by two dimensional x-ray image
CN108805974A (en) A kind of intensive non-rigid motion structure algorithm based on Grassmann manifold
Duan et al. Craniofacial reconstruction based on multi-linear subspace analysis
CN108280858A (en) A kind of linear global camera motion method for parameter estimation in multiple view reconstruction
Zhao et al. Application research of image recognition technology based on CNN in image location of environmental monitoring UAV
CN113255514B (en) Behavior identification method based on local scene perception graph convolutional network
Losada et al. Multi-camera sensor system for 3d segmentation and localization of multiple mobile robots
Marban et al. Estimation of interaction forces in robotic surgery using a semi-supervised deep neural network model
Salem et al. Three-dimensional pose estimation for laboratory mouse from monocular images
Danckaers et al. Building 3D statistical shape models of horticultural products
Madadi et al. Deep unsupervised 3D human body reconstruction from a sparse set of landmarks
Yang et al. SLAM Meets NeRF: A Survey of Implicit SLAM Methods
Firintepe et al. From IR images to point clouds to pose: point cloud-based AR glasses pose estimation
CN116079727A (en) Humanoid robot motion simulation method and device based on 3D human body posture estimation
Wu et al. Hi-net: boosting self-supervised indoor depth estimation via pose optimization
Li et al. Robust and efficient cpu-based rgb-d scene reconstruction
Khan et al. Skeleton based human action recognition using a structured-tree neural network
Habe et al. Pose estimation of swimming fish using NACA airfoil model for collective behavior analysis
Bercea et al. SHAMANN: Shared memory augmented neural networks

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication

Application publication date: 20181113

WW01 Invention patent application withdrawn after publication