CN109829972B - Three-dimensional human standard skeleton extraction method for continuous frame point cloud - Google Patents
Three-dimensional human standard skeleton extraction method for continuous frame point cloud Download PDFInfo
- Publication number
- CN109829972B CN109829972B CN201910056962.1A CN201910056962A CN109829972B CN 109829972 B CN109829972 B CN 109829972B CN 201910056962 A CN201910056962 A CN 201910056962A CN 109829972 B CN109829972 B CN 109829972B
- Authority
- CN
- China
- Prior art keywords
- skeleton
- frame
- point
- points
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention discloses a three-dimensional human standard skeleton extraction method for continuous frame point clouds, which comprises the following two steps: firstly, acquiring images of a multi-view moving human body, and reconstructing a dense point cloud model by using the images of each view of each frame. Performing downsampling and surface reconstruction on the point cloud model of each frame, and extracting a standard skeleton model by using a three-dimensional human body standard skeleton extraction algorithm based on model segmentation; secondly, carrying out interframe alignment and corresponding point matching on the extracted standard skeleton, constructing a skeleton point sequence of the continuous frame standard skeleton, establishing a continuous frame skeleton point position optimization model to optimize the skeleton point sequence obtained above, and finally obtaining the three-dimensional human body standard skeleton sequence facing the continuous frame point cloud.
Description
Technical Field
The invention relates to the field of computer graphics, in particular to research and development of a three-dimensional human standard skeleton extraction method for continuous frame point cloud.
Background
The skeleton frame extracted from the three-dimensional model can be regarded as a very visual and efficient representation of the model, and has strong practical significance for analyzing and operating the three-dimensional model. Especially for manikins, skeletal frames may be used for pose estimation, manikin, and manipulation.
In the two-dimensional group skeleton multi-granularity real-time extraction and tracking technology or in the three-dimensional kinect skeleton tracking data processing method, a standard skeleton with 20 points, which well reflects actual skeleton points of a human body, is used, as shown in fig. 2, but no human body standard skeleton extraction method without manual intervention exists at present.
Some of the existing skeleton extraction methods at presentIn the method, some technologies can directly extract the skeleton from incomplete point cloud models containing noise, such as skeleton model extraction technology based on generalized rotation symmetry axis concept and L of point cloud 1 Median skeleton extraction technology; there are some techniques that can extract the skeleton from the mesh model reconstructed from the point cloud model surface, such as skeleton model extraction techniques using average curvature flow, point cloud skeleton extraction techniques based on laplace shrinkage. However, for the three-dimensional human body model, the number of skeleton points in the skeleton extracted by the method is inconsistent, standardization cannot be achieved, the three-dimensional human body original model cannot be well attached, and in addition, the extracted three-dimensional human body skeleton occasionally has incomplete, wrong branches or partial point position deviation, as shown in fig. 1. Therefore, the human standard skeleton extracted by the invention has certain advantages in the aspects of integrity, correctness and standardization, and has certain practical value and significance in the aspects of subsequent animation production based on the skeleton, human operation and the like.
Disclosure of Invention
The invention provides a three-dimensional human standard skeleton extraction method for continuous frame point clouds. The method can extract the three-dimensional human standard skeleton with 20 skeleton points, which is more in line with the actual skeleton of the human body, from the continuous frame point cloud model under the condition of almost no manual intervention.
The technical scheme adopted by the invention is a three-dimensional human standard skeleton extraction method facing continuous frame point cloud, which comprises the following two steps: firstly, acquiring images of a multi-view moving human body, and reconstructing a dense point cloud model by using the images of each view of each frame. Performing downsampling and surface reconstruction on the point cloud model of each frame, and extracting a standard skeleton model by using a three-dimensional human body standard skeleton extraction algorithm based on model segmentation; secondly, carrying out inter-frame alignment and corresponding point matching on the extracted standard skeleton, constructing a skeleton point sequence of the continuous frame standard skeleton, establishing a continuous frame skeleton point position optimization model to optimize the skeleton point sequence obtained above, and finally obtaining a three-dimensional human body standard skeleton sequence facing the continuous frame point cloud, wherein the two steps are described in detail as follows:
firstly, a three-dimensional human body standard skeleton extraction method based on model segmentation.
Inputting a model: and shooting the person doing continuous motion in the light field at multiple visual angles, uniformly controlling each visual angle camera, and keeping a fixed frame rate. And then, performing dense matching based on the acquired multi-view images of the human body in motion and corresponding camera parameters generated by SfM by using a PMVS method, generating a three-dimensional point cloud model with dense human body, performing downsampling to reduce the number of point clouds so as to shorten the operation time of a subsequent algorithm, and further performing poisson surface reconstruction to finally generate a curved surface model, as shown in figure 3. It is worth noting that all models need to be normalized, aligning the frame models as much as possible. This preprocessing is crucial to the results of the method, since matching skeleton points in the standard skeleton of adjacent frames can be found more conveniently and accurately only if the size of each frame model is guaranteed to be uniform and the center position and orientation are in an identical coordinate system, and then in the optimization process.
S1.1, performing region segmentation on the three-dimensional human body model by using a model segmentation method, as shown in fig. 4.
Two grid models are input, an extended Mapper algorithm [ SMC07] is applied to generate shape graphs of the two grid models, and the corresponding relation between the two model segmentation areas is further obtained. The shape descriptor of each model described above is first computed as a set of real valued functions, which should capture the structure of the model while remaining unchanged for specific geometric details and robust to noise. In a specific implementation of the method, an HKS [ SOG09] function is used that intuitively indicates how close each point is to the end or high curvature region in the model. Then, the shape descriptor values of the two models are clustered together, each model is partitioned into regions in a joint way according to the shape descriptor values, a shape graph is constructed according to the clusters, each node of the shape graph corresponds to a region on the model, and the shape graph only retains the structure of the shape and no geometric information, namely only retains the topological structure information of the model. And then calculating the corresponding relation between the two shape diagrams so as to obtain the corresponding relation between the two model areas. In the method, two identical grid models are input, and finally, the region segmentation of the models and the topological structure information thereof are obtained and used by the subsequent steps.
S1.2, using a formula (1) to generate all region center points of the model as skeleton points, and generating an initial skeleton by connecting skeleton points by a model topological structure generated during region segmentation, as shown in fig. 5.
Where i represents the index of the segmented region of the current model, i.e., the index of the skeletal points, v i Representing the current bone point, v k Representing the current bone point v i Human body point cloud points of the segmented region, k represents the current bone point v i Human body point cloud point index of the located block, n represents bone point v i The number of human point clouds in the segmented region.
S1.3 filling or removing relevant skeleton points to generate a standard skeleton (20-point skeletons, 4 skeleton points on limbs, 1 skeleton point in the middle of crotch, 1 skeleton point in the waist, 1 skeleton point in the middle of shoulder, and 1 skeleton point in the head, as shown in figure 6, the following algorithm is used.
a) Inputting the step S1.2 to obtain an initial skeleton.
b) The method comprises the steps of judging a block where two arm center points are located and a block where shoulders are located by utilizing a special topological structure of a human body, performing secondary segmentation on the shoulders by utilizing distances between the shoulders and the two arm center points to generate three blocks, and finally further calculating the center points of the blocks after secondary segmentation of the shoulders as two shoulder points and a shoulder center point.
c) And extracting the standard skeleton based on the skeleton. The human body is divided into 6 blocks by utilizing a special human body structure that the number of connections between the shoulder center point and other bone points is 4 and the number of connections between the crotch center point and other bone points is 3, the number of points of each branch is calculated easily by utilizing the number of bone points in each block and the connection relation between the bone points and the tail ends (the tail ends points are provided with two hands, two feet and heads, and the number of connections between the tail ends and other bone points is S1.1), each branch is judged, and then the corresponding points are added or deleted to standardize the bone points on each block, so that the whole skeleton is standardized, and a specific algorithm is shown in fig. 7.
The optimization process of the second step is divided into two steps, firstly, the extracted standard skeleton is subjected to inter-frame alignment and inter-skeleton point matching, a skeleton point sequence of the continuous frame standard skeleton is constructed, and secondly, a continuous frame skeleton point position optimization model is constructed to optimize the continuous frame skeleton point positions:
s2.1 alignment and matching between frames of skeleton points of continuous frames. Because the head and the waist can be judged according to the number of points of each block in the standard skeleton and the connection relation between the point of the shoulder center and the point of the waist center, the interframe skeleton points of the head and the point of the waist skeleton can be easily realized, and the two arms and the two legs can be distinguished.
v i For the position of the point of the current block of the current frame, v' i For the positions of the corresponding points of the corresponding blocks of the adjacent frames, n represents the number of skeleton points in the current block and the corresponding block. The block with the smallest formula value in all possible corresponding relations is found to be the corresponding block, and the points in the corresponding block which are arranged in sequence are the corresponding points.
S2.2 continuous frame skeleton point position optimizing model
Continuous motion mannequins have many inter-frame geometric correlations, such as: the geometric positions of the bone points of the human body parts which are positioned at relative rest in the adjacent frames have local invariance, and the bone points of the human body parts of the adjacent frames reflecting the same action have the same and similar motion trail. Therefore, considering the point cloud geometrical correlation of adjacent frames for a three-dimensional human model of continuous motion state helps to extract more accurate bone points.
When representing and solving sparseness problems, L 0 Norms relative to L 1 、L 2 Norms are the best expression. In the field of computer graphics, L 0 Norms have been used to smooth and blur images, mesh model denoising, and point cloud model denoising. However, it is known that L is rarely used 0 The norms constrain the frame-to-frame constraint for the skeletal point position optimization process of the continuous frame standard skeleton. In addition, considering that the optimized skeleton points are likely to run out of the three-dimensional human body model, the extracted skeleton is meaningless, and L is utilized to avoid the occurrence of the conditions F The norms carry out intra-frame constraint on the optimized positions of the skeleton points, so that the optimized skeleton points are ensured to be near the central axis of the human body, and the occurrence of the condition that the skeleton points run out of a human body model in the optimization process is avoided to the greatest extent.
Thus based on L 0 Establishing an inter-frame sparse model based on L norms F The norm establishes an intra-frame constraint model, and solves the bone point positions which are related by motion through iterative optimization, so that the bone point positions are closer to the actual bone points of the human body.
And correcting and smoothing the generated standard skeleton points by utilizing the skeleton points corresponding to the adjacent frames and the position constraint of the adjacent skeleton points in the frames, and defining an optimization model shown in a formula (3).
In the above description, S is a skeleton point sequence matrix output after the skeleton point sequence of continuous frames is optimized, one row represents one frame, and correspondingly S 0 Representing a sequence of skeletal points of the successive frames initially input, alpha DS 0 The term represents the position constraint of the corresponding skeleton point between frames by using zero norm, so that the optimized skeleton between frames is smoother, wherein D is defined smoothing operator matrix, and each of the D is defined smoothing operator matrixAn element D mn And expressing the correlation of the m-th frame and the n-th frame, wherein alpha is a smoothing factor, controlling the inter-frame smoothing degree of the optimized skeleton points, and increasing alpha ensures that the correlation of the motion skeleton of each optimized frame is stronger, namely the skeleton sequence is smoother, but the motion characteristics of the skeleton of each frame can be reduced. Wherein D is defined as formula (4).
Wherein m, N represents the frame index of the skeleton, N in the above formula represents the total frame number of the skeleton sequence, |m-n|=1 and m, N is not equal to 1, N represents that the m-th frame and the N-th frame are adjacent frames, and neither the m-th frame nor the N-th frame is the head-to-tail frame.
Where S "is a movement range measure, each element S in The movement range of the i-th point in the nth frame is represented, and the definition is shown in formula (6).
Where n represents the frame index, i represents the current bone point index in the frame, j represents the bone point index in the same frame, |j-i|=1 represents the bone point index in the direction in which the i point is to be moved as j, and M is the bone point index that does not need to be moved. These coefficients are dynamically adjusted according to experimental data and need. To optimize the model of formula (3) above, an auxiliary variable δ is added, δ being a matrix, formula (3) becomes formula (7):
the variables S and delta are optimized in two steps respectively, and firstly S is kept unchanged to optimize delta, and the optimization problem is changed into a formula (8):
in solving this minimization problem, in
(D (i，:) Is the p-th row element of matrix D, S (:，j) Is the j-th element of matrix S, namely the j-th point of skeleton sequence, D (p，:) S (:，j) Inter-correspondence point association representing the jth bone point), let δ pj ＝0，δ pj The p-th row, j-th column element of matrix delta, otherwise delta pj ＝D (p，:) S (:，j) 。
After the solution of delta is completed, next, delta is fixed and unchanged, S is solved, and the minimization problem becomes formula (9):
equation (9) of the minimization problem is quadratic and therefore the minimum is found by derivation. After optimizing S, an iteration is completed, and the coefficient α=μ is updated α 、β＝μ β Re-optimizing by the above process until alpha reaches the threshold alpha max The optimization process ends.
And (3) outputting: standard skeleton sequences of three-dimensional manikins under successive frames.
On the premise of almost no manual intervention, the three-dimensional human skeleton extracted by the method has advantages over the skeleton extracted by the traditional method in terms of integrity, fitting degree with an original model, accuracy and standardization, and has practical value and significance.
Drawings
Fig. 1 is a skeleton extracted using an L1 median skeleton extraction method. Shown within the box are the anomaly locations.
FIG. 2 is a standard skeleton diagram. The standard skeleton with 20 points which reflects the actual skeleton points of the human body is better.
FIG. 3 is a process diagram of generating a surface model based on multi-view images. In the figure, the first column is a multi-view image acquired by a light field, the second column is sparse point cloud reconstruction based on the multi-view image, the third column is dense point cloud generation based on the obtained camera parameters and the sparse point cloud, the fourth column is a human body point cloud model without a birdcage, and the fifth column is a poisson surface reconstruction generation surface model of the dense point cloud.
Fig. 4 is a phantom segmentation.
FIG. 5 is an initial three-dimensional human skeleton
Fig. 6 is a three-dimensional human standard skeleton diagram. The four limbs of the 20-point skeleton are respectively provided with 4 skeleton points, the middle of the crotch is provided with 1 skeleton point, the waist is provided with 1 skeleton point, the middle of the shoulder is provided with 1 skeleton point, and the head is provided with 1 skeleton point.
Fig. 7 is a diagram of a standard skeleton algorithm.
Fig. 8 is a diagram of the matching effect of two adjacent frames of skeleton points.
Fig. 9 is a multi-view image of a moving human body acquired by a light field initially input in an experiment.
Fig. 10 is a comparison of the optimization of the bone point position by using the continuous frame under different data. In the figure, the first behavior is to input a model, the second behavior is to use a standard skeleton before optimizing a continuous frame skeleton point optimizing model and a model where the standard skeleton is located, and the third behavior is to use the standard skeleton after optimizing the continuous frame skeleton point optimizing model and the model where the standard skeleton is located.
FIG. 11 is an experiment with a conventional skeleton extraction methodResults are compared with a graph. The provided three-dimensional human body standard skeleton extraction method oriented to continuous frames is compared with the traditional classical skeleton extraction method in effect under different experimental data. In the figure, the human body model is input by the first action, the skeleton effect diagram is extracted by an average curvature skeleton extraction method proposed by Tagliosacchi and the like by the second action, the skeleton is extracted by a Laplacian shrinkage-based method of Cao and the like by the third action, the skeleton effect diagram is extracted by an L1 median skeleton extraction method of Huang and the like by the fourth action, and the skeleton effect diagram is extracted by an L-based method proposed by Zhang and the like by the fifth action 0 And (3) a skeleton optimization experimental effect diagram of the sixth behavior, and a standard skeleton effect diagram extracted by the method.
Detailed Description
The invention verifies the effectiveness of the invention for extracting the three-dimensional human standard skeleton through a comparison experiment with a similar method. Two main experiments are designed in the experimental part, namely the experiment one aims to verify the effectiveness of the continuous frame bone point position optimizing model; experiment II aims to verify the superiority of the skeleton extraction method in the invention compared with the traditional classical skeleton extraction method.
The experimental data set used in the present invention is as follows:
the invention uses 50 industrial cameras in a light field acquisition system to acquire multi-view color images of a moving human body, the pixels of each camera are about 220 ten thousand, the resolution of the acquired images is 2048 x 1088, and the specific acquisition information is as follows:
(1) The acquisition object is a walking man, as shown in fig. 9-a, the acquisition frame rate is 30 frames per second, the acquisition time period is 60 seconds, 1800 images are acquired in each view angle, and 50 different view angles are acquired.
(2) The acquisition object is a male who only performs arm unfolding motion, as shown in fig. 9-b, the acquisition frame rate is 30 frames per second, the acquisition duration is 60 seconds, 1800 images are acquired in each view angle, and 50 different view angles are acquired.
(3) The acquisition object is a male with arms and legs simultaneously moving, as shown in fig. 9-c, the acquisition frame rate is 30 frames per second, the acquisition time period is 60 seconds, 1800 images are acquired in each view angle, and 50 different view angles are acquired.
A number of experiments were performed on the continuous frame bone point location optimization model of the present invention using the data set described above. After the skeleton data of each frame of human body are obtained and corresponding skeleton standardization and skeleton point inter-frame alignment matching are carried out, the parameters in the table 1 are used for optimization, and a good result is obtained.
TABLE 1 optimization of parameters in model
| Parameters (parameters) | α | β | λ | α max | μ α | μ β |
| Initial value | 0.001 | 2.0 | 0.003 | 100 | 2 | 0.9 |
Experiment one: the effects of the continuous frame bone point location optimization model on the data set before and after optimization are compared as shown in fig. 10. In the figure, the first row represents an input model, the second row represents a standard skeleton before optimization by using a continuous frame skeleton point optimization model and a model where the standard skeleton is located, in order to ensure that the three-dimensional human standard skeleton extracted by the display method is more in line with an initial input human body model, the extracted standard skeleton and the input model corresponding to the extracted standard skeleton are put together for display, and the third row represents the standard skeleton after optimization by using the continuous frame skeleton point optimization model and the model where the standard skeleton is located.
The optimized model has good effect after optimization aiming at different human bodies and different motions. Fig. 10-a shows the comparison of the effects before and after optimization of a frame sequence with the problems of irregular skeleton, inability of the skeleton points to better fit the actual human body architecture, etc. before optimization. The optimization result shows that the skeleton obtained by optimizing the optimization model in the invention is more neat, better reflects the motion gesture of the human body, and is more suitable for the distribution of the actual human skeleton points. In fig. 10b-e, the number of frames of skeletal points is increased, and optimization experiments are performed on frame sequence models of different numbers of frames and with different motion postures. In fig. 10-e, the human skeleton starts extending horizontally from the arms, and the arms are bent obviously during the exercise from the end of the extension of the arms to the sides of the arms. In order to more clearly see the track in the motion process, 8 frames are selected for display. According to the figure, all skeleton points obtained through optimization of the optimization model are tidier and smoother than before optimization, and the overall movement trend is clearer and more obvious. And compared with the original point cloud model, the optimized skeleton points are better attached to the original model, the motion gesture of the original model is better reflected, and the distribution of the actual human skeleton points is better met.
The first experiment aims at verifying the effectiveness of the provided optimization model, and the skeleton optimized by the optimization model is closer to the real human skeleton, and is more neat, smoother and better in effect.
Experiment II: the proposed three-dimensional human standard skeleton extraction method oriented to continuous frames is compared with the effects of the traditional classical skeleton extraction method, as shown in fig. 11 a-c. The method for extracting the three-dimensional human body standard skeleton under the same data set without manual intervention has certain advantages compared with the traditional classical skeleton extraction method.
The first row as shown in the figure represents the input human model, the second row represents the skeleton effect map extracted by the average curvature skeleton extraction method proposed by Tagliosacchi et al, the third row represents the skeleton extracted by the Laplacian contraction-based method of Cao et al, the fourth row represents the skeleton effect map extracted by the L1 median skeleton extraction method of Huang et al, and the fifth row represents the L-based skeleton effect map proposed by Zhang et al 0 And (3) a skeleton optimization experimental effect diagram of the sixth behavior, and a standard skeleton effect diagram extracted by the method.
As can be seen from the figure, the skeleton extracted by the method is more complete, no false branches appear, the human body posture is better reflected, the skeleton points are consistent and more standard, and the skeleton is a standard skeleton with 20 skeleton points which better reflects the actual skeleton point positions of the human body. The three-dimensional human body standard skeleton extraction algorithm based on model segmentation can extract a three-dimensional human body standard skeleton with 20 points, and the constraint of the positions of the skeleton points among frames and the constraint of the positions of the skeleton points in frames when the continuous frame skeleton point position optimization algorithm optimizes the skeleton points make the final standard skeleton more accurate, tidier and more fit with an original input model, and more fit with the distribution of the skeleton points of an actual human body. Therefore, the method provided by the invention has better effect than the traditional skeleton extraction method, and is more convenient to be used by subsequent posture estimation, human body modeling, operation and the like.
Claims (1)
1. A three-dimensional human standard skeleton extraction method for continuous frame point cloud is characterized in that: the method comprises the following two steps: firstly, acquiring images of a multi-view moving human body, and reconstructing a dense point cloud model by using the images of each view of each frame; performing downsampling and surface reconstruction on the point cloud model of each frame, and extracting a standard skeleton model by using a three-dimensional human body standard skeleton extraction algorithm based on model segmentation; secondly, performing inter-frame alignment and corresponding point matching on the extracted standard skeleton, constructing a skeleton point sequence of the continuous frame standard skeleton, establishing a continuous frame skeleton point position optimization model to optimize the skeleton point sequence obtained above, and finally obtaining a three-dimensional human body standard skeleton sequence facing the continuous frame point cloud;
the process of optimizing the bone site location of successive frames is as follows,
firstly, realizing interframe alignment and interframe matching of extracted standard frameworks, constructing a skeleton point sequence of a continuous frame standard framework, and secondly, constructing a continuous frame skeleton point position optimizing model to optimize the continuous frame skeleton point positions:
s2.1, aligning and matching between continuous frame skeleton point frames; realizing that the interframes of the two arms respectively correspond to the interframes of the two legs respectively:
v i for the position of the point of the current block of the current frame, v' i N represents the number of skeleton points in the current block and the corresponding block for the positions of the corresponding points of the corresponding blocks of the adjacent frames; finding out the block with the smallest formula value in all possible corresponding relations to be regarded as the corresponding block, wherein the points in the corresponding block which are arranged in sequence are corresponding points;
s2.2 continuous frame skeleton point position optimizing model
The continuous motion human body model has multi-frame geometric correlation, the geometric positions of the bone points of the human body parts which are positioned at relative rest in adjacent frames have local invariance, and the bone points of the human body parts of the adjacent frames reflecting the same action have the same and similar motion trail; aiming at a three-dimensional human body model in a continuous motion state, the point cloud geometric correlation of adjacent frames is considered, so that more accurate bone points can be extracted;
by L F The norms carry out intra-frame constraint on the optimized positions of the bone points, ensure that the optimized bone points are near the central axis of the human body, and furthest avoid the bone points from escaping from the human body model in the optimizing processOccurrence of a condition;
thus based on L 0 Establishing an inter-frame sparse model based on L norms F Establishing an intra-frame constraint model by using norms, and solving the positions of skeleton points which are related by motion through iterative optimization, so that the positions are closer to actual skeleton points of a human body;
correcting and smoothing the generated standard skeleton points by utilizing the skeleton points corresponding to the adjacent frames and the position constraint of the adjacent skeleton points in the frames, and defining an optimization model shown in a formula (3);
wherein S is a skeleton point sequence matrix output after skeleton point sequence optimization of continuous frames, one row represents one frame, and correspondingly S 0 Representing a sequence of skeletal points of the successive frames initially input, alpha DS 0 The term represents the position constraint of the corresponding skeleton point between frames by using zero norm, so that the optimized skeleton between frames is smoother, wherein D is defined smoothing operator matrix, and each element D mn Representing the association of the m-th frame and the n-th frame bones, wherein alpha is a smoothing factor, and controlling the inter-frame smoothing degree of the optimized bone points; wherein D is defined as formula (4);
wherein m, N represents the frame index of the skeleton, N in the above formula represents the total frame number of the skeleton sequence, |m-n|=1 and m, N is not equal to 1, N represents that the m-th frame and the N-th frame are adjacent frames, and neither the m-th frame nor the N-th frame is the head-to-tail frame;
where n represents the frame index, i represents the current bone point index in the frame, j represents the bone point index in the same frame, |j-i|=1 represents the bone point index in the direction in which the i point is to be moved as j, and M is the bone point index that does not need to be moved; dynamically adjusting the coefficients according to experimental data and needs; to optimize the model of formula (3) above, an auxiliary variable δ is added, δ being a matrix, formula (3) becomes formula (7):
the variables S and delta are optimized in two steps respectively, and firstly S is kept unchanged to optimize delta, and the optimization problem is changed into a formula (8):
in solving this minimization problem, in
Is the p-th row element of matrix D, S (：，j) Is the j-th element of matrix S, namely the j-th point of skeleton sequence, D (p，：) S (：，j) Let delta when representing inter-correspondence point association of jth bone point pj ＝0，δ pj The p-th row, j-th column element of matrix delta, otherwise delta pj ＝D (p，：) S (：，j) ；
After the solution of delta is completed, next, delta is fixed and unchanged, S is solved, and the minimization problem becomes formula (9):
the equation (9) for the minimization problem is quadratic and the minimum is found by derivation; after optimizing S, an iteration is completed, and the coefficient α=μ is updated α 、β＝μ β Re-optimizing by the above process until alpha reaches the threshold alpha max And (3) ending the optimization process;
and (3) outputting: standard skeleton sequences of three-dimensional manikins under successive frames.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910056962.1A CN109829972B (en) | 2019-01-19 | 2019-01-19 | Three-dimensional human standard skeleton extraction method for continuous frame point cloud |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910056962.1A CN109829972B (en) | 2019-01-19 | 2019-01-19 | Three-dimensional human standard skeleton extraction method for continuous frame point cloud |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109829972A CN109829972A (en) | 2019-05-31 |
| CN109829972B true CN109829972B (en) | 2023-06-02 |
Family
ID=66861649
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910056962.1A Active CN109829972B (en) | 2019-01-19 | 2019-01-19 | Three-dimensional human standard skeleton extraction method for continuous frame point cloud |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109829972B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110363862B (en) * | 2019-07-15 | 2023-03-10 | 叠境数字科技（上海）有限公司 | Three-dimensional grid sequence compression method based on human body template alignment |
| CN111274909B (en) * | 2020-01-16 | 2022-05-20 | 重庆邮电大学 | Human body point cloud framework extraction method based on deep learning |
| CN111274998B (en) * | 2020-02-17 | 2023-04-28 | 上海交通大学 | Parkinson's disease finger knocking action recognition method and system, storage medium and terminal |
| CN111738220B (en) * | 2020-07-27 | 2023-09-15 | 腾讯科技（深圳）有限公司 | Three-dimensional human body posture estimation method, device, equipment and medium |
| CN112884822B (en) * | 2021-02-09 | 2024-03-15 | 北京工业大学 | Framework extraction method of human body multi-view image sequence based on RepNet model |
| CN113140007B (en) * | 2021-05-17 | 2023-12-19 | 上海驭矩信息科技有限公司 | Concentrated point cloud-based set card positioning method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102467753A (en) * | 2010-11-04 | 2012-05-23 | 中国科学院深圳先进技术研究院 | Method and system for reconstructing time-varying point cloud based on framework registration |
| CN106204635A (en) * | 2016-06-27 | 2016-12-07 | 北京工业大学 | Based on L0the human body successive frame skeleton optimization method minimized |
| CN108122275A (en) * | 2017-12-22 | 2018-06-05 | 清华大学 | Dynamic realtime 3 D human body method for reconstructing and system based on skeleton tracking |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8023726B2 (en) * | 2006-11-10 | 2011-09-20 | University Of Maryland | Method and system for markerless motion capture using multiple cameras |
| CN105005995B (en) * | 2015-07-29 | 2017-07-25 | 武汉大学 | A kind of method for calculating three-dimensional point cloud model bone |
| CN108154551B (en) * | 2017-11-29 | 2021-04-30 | 奥比中光科技集团股份有限公司 | Method and system for dynamically reconstructing three-dimensional human body model in real time |
| CN108711185B (en) * | 2018-05-15 | 2021-05-28 | 清华大学 | Three-dimensional reconstruction method and device combining rigid motion and non-rigid deformation |
-
2019
- 2019-01-19 CN CN201910056962.1A patent/CN109829972B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102467753A (en) * | 2010-11-04 | 2012-05-23 | 中国科学院深圳先进技术研究院 | Method and system for reconstructing time-varying point cloud based on framework registration |
| CN106204635A (en) * | 2016-06-27 | 2016-12-07 | 北京工业大学 | Based on L0the human body successive frame skeleton optimization method minimized |
| CN108122275A (en) * | 2017-12-22 | 2018-06-05 | 清华大学 | Dynamic realtime 3 D human body method for reconstructing and system based on skeleton tracking |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109829972A (en) | 2019-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109829972B (en) | Three-dimensional human standard skeleton extraction method for continuous frame point cloud | |
| Xu et al. | Ghum & ghuml: Generative 3d human shape and articulated pose models | |
| Jiang et al. | Selfrecon: Self reconstruction your digital avatar from monocular video | |
| Alldieck et al. | imghum: Implicit generative models of 3d human shape and articulated pose | |
| Zhang et al. | Interacting two-hand 3d pose and shape reconstruction from single color image | |
| CN109636831B (en) | Method for estimating three-dimensional human body posture and hand information | |
| Stoll et al. | Fast articulated motion tracking using a sums of gaussians body model | |
| Balan et al. | Detailed human shape and pose from images | |
| Li et al. | Markerless shape and motion capture from multiview video sequences | |
| Wang et al. | Sequential 3D human pose and shape estimation from point clouds | |
| Lei et al. | Cadex: Learning canonical deformation coordinate space for dynamic surface representation via neural homeomorphism | |
| Stoll et al. | Template Deformation for Point Cloud Fitting. | |
| US20230169727A1 (en) | Generative Nonlinear Human Shape Models | |
| CN106204635B (en) | Based on L0The human body successive frame bone optimization method of minimum | |
| WO2021063271A1 (en) | Human body model reconstruction method and reconstruction system, and storage medium | |
| CN112330813A (en) | Wearing three-dimensional human body model reconstruction method based on monocular depth camera | |
| CN115951784A (en) | Dressing human body motion capture and generation method based on double nerve radiation fields | |
| Han et al. | Robust shape estimation for 3D deformable object manipulation | |
| Venkat et al. | HumanMeshNet: Polygonal mesh recovery of humans | |
| Taylor et al. | VR props: an end-to-end pipeline for transporting real objects into virtual and augmented environments | |
| Madadi et al. | Deep unsupervised 3D human body reconstruction from a sparse set of landmarks | |
| Jiang et al. | Observation-oriented silhouette-aware fast full body tracking with Kinect | |
| CN116310066A (en) | Single-image three-dimensional human body morphology estimation method and application | |
| CN115346000A (en) | Three-dimensional human body reconstruction method and device, computer readable medium and electronic equipment | |
| Chen et al. | Optimizing human model reconstruction from RGB-D images based on skin detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |