CN102129719A - Virtual human dynamic model-based method for extracting human skeletons - Google Patents

Abstract

The invention discloses a virtual human dynamic model-based method for extracting human skeletons, which is mainly used for automatically extracting the human skeletons through vertex clustering. The method comprises the following steps of: firstly, selecting one posture as a reference posture, calculating a transformation matrix of a corresponding triangular facet between other postures and the reference posture with the triangular facet as a base unit, then converting the transformation matrix into a multidimensional vector, clustering the multidimensional vector, blocking a human body, and solving center points of block joints; and then solving five characteristic points of a head, two hands and two feet of the human body by utilizing a geodesic distance and human physiological characteristics, then sequentially connecting the center points of the block joints starting from the five characteristic points so as to form the main skeletons, such as palms, lower arms, upper arms, foot soles, legs, thighs, the head, a trunk and the like of the human body, wherein the center points of the block joints are human articulation points.

Description

Skeleton extracting method based on visual human's dynamic model

Technical field

The present invention relates to a kind of skeleton extracting method of visual human's dynamic model, be mainly used in computing machine human body animation.

Background technology

The virtual human body cartoon technique is the important component part of virtual reality technology, mainly is to utilize computer graphical, image technique, drives three-dimensional (3 D) manikin and moves in conjunction with kinematics, dynamic method.It mainly contains the mode of two kinds of model animations, summit animation and skeleton cartoon.In the animation of summit, every frame animation is exactly a particular pose of manikin in fact.By the method for interpolation between frame, obtain level and smooth animation effect.In skeleton cartoon, the skeleton structure that manikin has interconnected " bone " forms, is that model generates animation by the change bone towards coming with the position.Skeleton cartoon compare with the summit animation take up room littler, because it does not need will store the data on each summit of each frame model as the animation of summit, but only need store a cover bone, by the binding of model vertices and bone, drive human movement model.In the manufacturing process of 3 D human body skeleton cartoon, need artificial skeletal structure to three-dimensional (3 D) manikin to mark, manual mark task is too heavy, and too responsive to model attitude.Therefore the present invention utilizes computer technology that the skeletal structure of visual human's model is extracted automatically.

Visual human's model bone extracts, and refers to by the analysis to geometric elements such as the point of component model, limit, faces, and in conjunction with the dissect physiology characteristics and the movement characteristic of human body, the articulation point of extraction model generates the bone of manikin.Because the complicacy and the scrambling of manikin, and the diversity of human body attitude variation make that the automatic extraction of model bone is very difficult.What occur at present extracts the method for bone from manikin, can be divided into following two classes according to the difference of required model:

(1) based on the method for static model

Static model are meant the manikin that has only an expression human body a certain particular pose, based on the algorithm of static model by the geometrical Characteristics Analysis of model is extracted the articulation point position.Joao is (referring to document 1-Oliveira J, Zhang D, Spanlang B, Buxton B.Animating scanned human models.Journal of WSCG, 2003,11 (2): 362-369.) wait the people to utilize the horizontal section information of scan model, the center line of extraction model, essential characteristic point (under oxter, the left and right sides, the hip) according to reentrancy algorithm human body bifurcation, extract the body local unique point relevant according to local curvature with articulation center, with on the center line with the nearest point of local feature point as articulation center.Ju is (referring to document 2-Ju Xiangyang, Werghi Naoufel, Siebert J Paul.Automatic segmentation of 3D human body scans[C] //Proc of the Computer Graphics and Imaging.Las Vegas:[s.n.], 2000) utilize one group of horizontal section intercepting manikin, according to each layer cross section information model is divided into four limbs and first-class five parts of chest, perimeter change rule based on each part middle section elliptic contour extracts the articulation point that each part comprises.The algorithm of document 1,2 all adopts the horizontal section truncated models, and its result of calculation changes with the change of factors such as model attitude, the bodily form, when manikin attitude and standard attitude differ big, can't obtain correct topological structure; Secondly, extract the feature (as profile girth, mean radius) that articulation center adopted, not enough robust influenced by model accuracy, thereby result of calculation is not accurate enough.Werghi is (referring to document 3-Werghi N Yijun Xiao, Siebert J P.A functional2based segmentation of human body scans in arbit rary postures[J] .IEEE Trans on Systems, Man, and Cybernetics, Part B:Cybernetics, 2006,36 (1): 1532165) etc. the people with geodesic line distance between the human body surface each point as the Morse function, constructed discrete Reeb figure (discrete Reeb graph, DRG), realized the piecemeal of any attitude scanning manikin according to the priori of this figure and anthropometry.But this algorithm has only been finished the piecemeal of manikin, does not realize the extraction of articulation center.Domestic in brave (referring to document 4-in brave Wang Zhao during its summer Hong Maotian reveal. the manikin bone extracting method that a kind of attitude is irrelevant. computer research and development, 2008,45 (07): 1249-1258) wait the same method acquisition human body topological structure that adopts based on geodesic distance of people, carry out location, bone joint by profile like circle property function, can realize the joint extraction that attitude is irrelevant.Though this algorithm is based on the hypothesis of human anatomy, the juxtra-articular cross sectional shape of model presents irregularly shaped, and the bone middle part presents class circle property, but because the cross section of this algorithm is from the beginning to back down the beginning to calculate geodesic distance, therefore there are deviation in cross section profile and medical science indication cross section, its anti-interference is poor, is subject to influences such as attitude, model accuracy.

(2) based on the method for dynamic model

Dynamic model is made up of the animation sequence of the different attitudes of the same human body of expression, the corresponding one group of vertex data of each attitude, roughly study thinking and be and select one of them attitude for reference to attitude, with face, limit or summit is base unit, calculating is with reference to variable quantity or the transformation matrix of attitude between other attitude, based on this variable quantity or transformation matrix cluster is carried out on summit or face, determine the articulation point position in abutting connection with situation according to class.Compare with static model, because data volume is big, the information that can obtain is also many, though can be higher than the extracting method of static model computing time, it is high that precision is wanted.(referring to document 5-Kirk A.G., O ' Brien J.F., Forsyth D.A.:Skeletal parameter estimation from optical motion capture data.In CVPR 2005 (2005), pp.782-788; Referring to document 6-Edilson de Aguiar, Christian Theobalt, Sebastian Thrun, Hans-Peter Seidel, Automatic Conversion of Mesh Animations into Skeleton-based Animations.EUROGRAPHICS 2008, Volume 27 (2008), and Number 2; Referring to document 7-Schaefer S., Yuksel C.:Example-based skeleton extraction.In SGP ' 07 (Aire-la-Ville, Switzerland, 2007), pp.153-162.) but these methods need be carried out loaded down with trivial details data pre-service such as summit registration between different attitudes, and can the articulation point locating accuracy reflect fully that with animation data the motion conditions in each joint is relevant.

In a word, the problem of method existence at present is: based on static model, owing to have only an attitude, lack the movable information in joint, only with geological information and a simplex criteria of model, be difficult to accurately extraction articulation point position; Based on dynamic model, require the animation sequence of input model, can the articulation point locating accuracy reflect fully that with animation data the motion conditions in each joint is relevant.

Summary of the invention

The technical problem to be solved in the present invention: overcome the deficiencies in the prior art, propose a kind of human body priori that makes full use of, based on the skeleton extracting method of visual human's dynamic model and human body geodesic distance.This method can realize the automatic extraction of the manikin bone of a plurality of different attitude model sequences of same visual human, has stronger robustness, owing to the use of Mean-Shift clustering algorithm, improved the accuracy of extracting method computing velocity and human synovial point location.

Description of drawings

The process flow diagram that Fig. 1 adopts for the present invention;

Fig. 2 is algorithm substep result of calculation figure, and Fig. 2 a is one group of dynamic model sequence; The standard attitude model of Fig. 2 b for choosing arbitrarily; Fig. 2 c carries out clustering result figure for utilizing the Mean-Shift algorithm to triangular facet; Fig. 2 d is for asking for the figure as a result of 5 human body feature points to the good standard attitude model of cluster; Fig. 2 e is that the human body skeletal structure finally extracts figure as a result;

Fig. 3 is articulation point location and bone connection layout; Fig. 3 a is human body piecemeal and articulation point location map, and Fig. 3 b is the bone line graph;

Fig. 4 is the procedure chart of Mean-Shift algorithm cluster two-dimensional points; Fig. 4 a is the sample point that 2D is scattered; Fig. 4 b is sample point cluster result figure;

Fig. 5 is a human body feature point extraction algorithm synoptic diagram;

Fig. 6 is the used department pattern of experiment; Fig. 6 a is the Dance mode set; Fig. 6 b is the SCAPE mode set;

Fig. 7 bone extracts experimental result picture.

Embodiment

Skeleton based on geodetic survey model extracts and orientation judging method, it is characterized in that may further comprise the steps:

(1) reference model is chosen

As input, each grid model is one group of vertex data, has preserved the position and the communication information on each summit with the grid model of a plurality of different attitudes of same visual human in the present invention.The present invention with the model of any one of them attitude as the reference model, for step is later on prepared.

(2) multi-C vector and the cluster of structure triangular facet

Because during visual human's mesh motion, usually the triangular facet sector-meeting that belongs to same bone control has similar transformation matrix, therefore at first to choose any one model according to the description of step (1) from the model sequence of a plurality of different attitudes be reference model in the present invention, with the triangular facet is base unit, calculate other attitude model to transformation matrix with reference to triangular facet corresponding between the attitude model, afterwards this transformation matrix is converted into the sample data of multi-C vector, realizes the human body piecemeal thereby these triangular facets are carried out cluster as next step clustering algorithm.

The triangular facet rotatable sequence is to determine that like this three summits of at first establishing a triangular facet are respectively i ₁, i ₂And i ₃, the nonopiate matrix that is defined in t attitude intermediate cam face j afterwards is Wherein

Be the i of t attitude model _aThe position of point.The same nonopiate matrix of establishing with reference to attitude intermediate cam shape j is

Deformation matrix from triangular facet Utilize polar coordinates to decompose in (in the t attitude), Extract the triangle rotation matrix

(the R here is the abbreviation of Rotation, represents rotation matrix (rotation matrix in the graphics is all represented with R usually), and subscript t is a t model, and subscript j is a j triangular facet of t model), here

It is a symmetric matrix.

After obtaining 3 * 3 relative rotation matrixs of triangular facet j, can make up the vector Z of a dimension d=9S _j∈ R ^9S(S is the model number), this vector is to be combined by rotatable sequence,

Here vec (R): R ^{3 * 3}→ R ⁹, be about to one 3 * 3 rotation matrix R ^{3 * 3}Be converted to one 9 dimensional vector R ⁹Afterwards just can be to this rotatable sequence point { Z _jCarry out the Mean-Shift cluster, thus the triangular facet that will have close variation carries out cluster.Gather and promptly belong to same human body piecemeal for of a sort triangular facet.

(3) articulation point location and bone extract

After utilizing the Mean-Shift algorithm that visual human's model is carried out piecemeal, the bone piece number of definite visual human's model that just can summary, the central point of the boundary surface of each piecemeal may be exactly potential human joint points simultaneously.But this moment, we can not distinguish the physiological significance of every bone, and promptly we can't judge which piece bone represents arm, and which piece bone is represented trunk or shank etc.Therefore we need following steps to obtain human body articulation point position and skeletal structure accurately:

A. each divides the central point c of interblock to obtain visual human's model _iClustering algorithm of the present invention before is that base unit carries out with the triangular facet, so the present invention can give a color value with each leg-of-mutton each summit according to the difference of cluster.Shown in Fig. 3 (a), the full assignment in all summits that with cluster is foot earlier is a pink colour, and when the present invention then is redness to all summit assignment that belong to shank, find to be assigned pink colour before some summit, these summits are the summit that two piecemeals have a common boundary, and the mean place of obtaining these summits is candidate's bone central point c _i, identify c simultaneously _iThe color value of point is PINK and RED.

B. be input with human body with reference to attitude, obtain five unique points in the manikin: a head unique point p _Head, two hand-characteristic point p _Arm1, p _Arm2, two unique point p of foot _Leg1, p _Leg2

C. connect each articulation point and form bone.Shown in Fig. 3 b, the first step is tried to achieve the unique point p of foot _Leg1, through second step processing back p _Leg1Will be assigned pink colour (PINK), find c simultaneously _AnkleThe sign color of point also has PINK, therefore with p _Leg1With c _AnkleLink to each other and form a bone; C in like manner _AnkleWith c _KneeSimultaneously underlined is the color of RED, and therefore these two points being linked to each other forms another bone.By that analogy,, connect each classification center point successively, just formed the human skeleton structure respectively since 5 unique points.

Technical solution of the present invention is characterized in that utilizing the Mean-Shift algorithm that the rotatable sequence of triangular facet is carried out cluster:

The present invention carries out cluster with the Mean-Shift algorithm to the rotatable sequence of triangular facet, the Mean-Shift algorithm is a kind of iteration clustering algorithm of sane practicality, its mode with iteration moves (shift) to each data point to average (mean) position of neighborhood, therefore we are classified as a class to the data point that can move to same density peaks, realize cluster with this.

Given n point, the i.e. vector Z formed of the triangular facet rotatable sequence that calculates of step (2) _j∈ R ^d, the Mean-Shift algorithm of stationary window radius is set up the cuclear density function,

Here, k () is the kernel function of radial symmetry, and h is the nuclear windows radius.Just can calculate skew average point and the skew mean vector of this density function f (z) afterwards.

Because the dimension of the data of this paper algorithm institute cluster is 9S (S is the number of the manikin of the different attitudes of input), the cluster process of describing high dimension vector with chart is not very clear, so this paper is scattered with two dimension and represents the triangular facet rotatable sequence of 9S dimension to carry out the explanation of Mean-Shift cluster process.As shown in Figure 4:

The 2 dimension sample points that are scattered among Fig. 4 a are represented the rotatable sequence of visual human's model intermediate cam face, by the Mean-Shift algorithm these sample points are carried out cluster, and poly-have identical color for of a sort sample point.Fig. 4 b has shown the track that each point rises according to Mean-Shift algorithm gradient, red point has been represented the final mean place of each classification, the sample point that respectively is scattered that is to say if can carry out mean shift according to the track that gradient rises, finally be displaced to the peak point of same density, being about to these sample points poly-is a class.Just these triangular facets being gathered is a class, thinks that promptly they are subjected to the influence of same bone.Shown in Fig. 2 c, the triangular facet with same color is same human body piecemeal to the cluster result of triangular facet in the present invention.

Technical solution of the present invention is characterized in that having proposed extracting and recognition methods with the irrelevant human body feature point of human body attitude:

Aspect of model point is meant the summit that is positioned at head and limb end, and we will be the geodesic distance that starting point is calculated each articulation point with these unique points, realizes the articulation point location.At first appoint and get a bit at model surface, calculate the geodesic distance that this puts other summit of model surface, geodesic distance is got peaked point as first unique point, similarly, be starting point with this unique point again, geodesic distance is got at peaked o'clock as second unique point, when asking third and fourth unique point, will arrive the geodesic distance of the unique point of having tried to achieve and get peaked as Next unique point.When extracting the 5th unique point, increased by one to extract minutiae distance greater than the constraint condition of certain value, to prevent getting a plurality of unique points at same position.With g (x, y) geodesic distance between expression summit x and the y.Specific algorithm is as follows:

1. initialization

Feature point set

2. calculate first unique point p ₁

$\∀v^{\′}\∈M$

$p_1=\underset{v_i}{\arg \max }\left(g(v^{\′},v_i)\right),v_i\∈M$

P＝P∪{p ₁}

3. calculate second unique point p ₂

${\mathrm{<figure-callout\; id="2"\; label="p"\; filenames="HDA0000050719630000012.png"\; state="\{\{state\}\}">p</figure-callout>}}_2=\underset{v_i}{\arg \max }\left(g(p_1,v_i)\right),v_i\&Element;M$

P＝P∪{p ₂}

g _min＝αg(p ₁，p ₂)

4. find the solution p ₃, p ₄

For?i＝3?to?4

$p_i=\underset{v_i}{\arg \max }\left(\underset{j=1}{\overset{i-1}{\mathrm{\&Sigma;}}}g(p_j,v_i)\right),v_i\&Element;M,\&ForAll;p_j\&Element;P$

P＝P∪{p _i}

End?for

5. find the solution p ₅

$p_5=\underset{v_{\text{i}}}{\arg \max }\left(\underset{j=1}{\overset{4}{\mathrm{\&Sigma;}}}g(p_j,v_i)\right),v_i\&Element;M$ And g (v _i, p _j)＞g _Min, $\&ForAll;p_j\&Element;P$

This algorithm is at first set the point set P of a sky, appoint afterwards get in visual human's model a vertex v ', try to achieve some p apart from v ' some geodesic distance maximum ₁, p ₁Be first unique point of visual human's model, according to the Human physiology priori, this point must be positioned at the top of hand or foot.Fig. 5 is a feature point extraction algorithm synoptic diagram.

As shown in Figure 5, the unique point p that at first tries to achieve ₁Be positioned at the hand top; Afterwards with first unique point p ₁Be starting point, computed range p ₁The point of some geodesic distance maximum is second unique point p ₂If, p ₁Be positioned at the hand (foot) of human body, so p ₂Then must be positioned at foot's (hand), p among Fig. 5 ₂Be second unique point of being tried to achieve; Try to achieve apart from p afterwards ₁, p ₂The point p of geodesic distance sum maximum ₃, p ₃Be the 3rd unique point; The 4th unique point p ₄For arriving p ₁, p ₂, p ₃The point of some geodesic distance sum maximum; If the 5th unique point p ₅Still adopt third and fourth unique point ask method the time, can find that the 5th unique point of being tried to achieve is positioned at foot simultaneously, as the v among Fig. 5 ₁Therefore point need add constraint condition, i.e. p when trying to achieve the 5th unique point ₅Distance to other unique points is greater than a threshold value g _Min, g _MinFormula be α g (p ₁, p ₂), experimental result shows that the value of α is that 0.25 to 0.35 effect is better here.After adding constraint condition, just can accurately obtain the 5th unique point p ₅

After trying to achieve five unique points, by the Human physiology feature as can be known a geodesic distance in one's hands less than any geodesic distance between other two unique points, be hand hand, trick, pin, geodesic distance between the pin pin all is greater than the geodesic distance between a hand, and the symmetry according to manikin just can identify head again, the unique point of hand and foot, at last according to homonymy trick distance less than heteropleural trick distance, can judge whether the trick unique point is positioned at homonymy or heteropleural.Arthmetic statement is as follows:

1. appoint among the feature point set P and get two unique points and make their geodesic distance minimum

d _first?min＝arg?min(g(p _i，p _j))，p _i，p _j∈P

2. from feature point set, appoint again and get two unique points and make their geodesic distance second little, promptly be only second to d _{First min}

d _sec?ond?min＝arg?sec?ond?min(g(p _k，p _t))，p _k，p _t∈P

3. try to achieve head feature point p according to the manikin symmetry _HeadAnd two hand-characteristic point p _Arm1, p _Arm2

IF(p _i＝＝p _k)

p _head＝p _i＝p _k；p _arm1＝p _j；p _arm2＝p _t；

ELSEIF(p _i＝＝p _t)

p _head＝p _i＝p _t；p _arm1＝p _j；p _arm2＝p _k；

ELSEIF(p _j＝＝p _k)

p _head＝p _j＝p _k；p _arm1＝p _i；p _arm2＝p _t；

ELSE

p _head＝p _j＝p _t；p _arm1＝p _i；p _arm2＝p _k；

4. two other unique point is the unique point p of foot among the feature point set P _Leg1, p _Leg2

5. be positioned at homonymy or heteropleural less than the heteropleural trick apart from each unique point of judgement according to homonymy trick distance

IF(g(p _arm1，p _leg1)＜g(p _arm1，p _leg2))

p _Arm1With p _Leg1Homonymy; p _Arm2With p _Leg2Homonymy

ELSE

p _Arm1With p _Leg2Homonymy; p _Arm2With p _Leg1Homonymy

Test findings of the present invention:

Data model of the present invention comprises the SCAPE manikin of Stanford University and the Dance manikin of Massachusetts Institute of Technology (MIT), and wherein SCAPE and Dance are made up of the human body of 71 and 200 different gestures respectively, and Fig. 6 has shown department pattern wherein.All experiments are all at a P4 3.0GHz, in save as on the PC of 2GB and carry out, programmed environment is Windows visual c++ .net 2005 environment.All manikins that experiment is adopted constitute by triangle gridding, and model dough sheet number is between 14000 to 25000, and number of vertex is between 7000 to 12500.

(1) feature point extraction

For feature point extraction, selected totally 272 on the different attitude models of different human body model for use with same human body, test result is found the algorithm that employing this paper provides, and has all found five unique points that are positioned at hand, pin, head position.Aspect unique point identification because head, between the hand unique point geodesic distance from be significantly less than between other any two unique points geodesic distance from, therefore, all 272 tested models can be gone out each unique point position by accurate recognition.But judging the trick unique point whether during homonymy, for some attitude, the geodesic distance defection between the unique point of homonymy trick wherein has 3 examples in 200 Dance models greater than the geodesic distance between the unique point of heteropleural trick, in 71 SCAPE models 14 examples is arranged.

(2) articulation point location

The present invention about the experiment of articulation point location with the Dance model grid model of totally 200 attitudes as the input data, four groups of experiments have been designed, every group of experiment all is to randomly draw 30 models to import as algorithm from 200 human body attitude models, emphatically to verified by the bigger wrist joint of attitude variable effect, elbow joint, ankle joint, knee joint.

Experimental group 1 all is to appoint from the different attitude models of 200 Dance human bodies at random that to get 30 be one group to experimental group 4, and dynamic human body model bone extraction algorithm is experimentized.Distinguished different human body piecemeals with different colours among Fig. 7, all there is identical rotational transform on the summit in each color piecemeal with triangular facet.The central point of trying to achieve different human body piecemeal junctions again is visual human's articulation point.Table 1 compares for the position difference of each group experiment articulation point result of calculation with true articulation point.

And utilize existing manikin bone extraction algorithm such as Zhou Changfa and like the average error of circle rate method articulation point location at 4cm to 5cm, and utilize method average error of the present invention to be lower than 2cm.This shows that the information that dynamic model of the present invention can extract is many, it is also more accurate that calculate in corresponding joint.

The joint	First group	Second group	The 3rd group	The 4th group
					Wrist	1.6cm	1.4cm	2.2cm	1.7cm
Ancon	1.1cm	1.5cm	1.3cm	1.6cm
					Ankle	2.1cm	1.8cm	2.0cm	1.9cm
Knee	1.7cm	1.7cm	1.5cm	1.4cm

The part that position difference comparison the present invention of each group experiment articulation point result of calculation of table 1 and true articulation point does not elaborate belongs to technology as well known to those skilled in the art.

Claims (3)

1. based on the skeleton extracting method of visual human's dynamic model, it is characterized in that may further comprise the steps:

(1) with the grid model of a plurality of different attitudes of same visual human as input, choosing one of them grid model arbitrarily is reference model;

(2) triangular facet with visual human's grid model is a unit, and calculating other attitude model except that reference model is converted into multi-C vector to the transformation matrix of triangular facet corresponding between reference model with described transformation matrix; By described multi-C vector is carried out cluster human body is carried out piecemeal;

(3) utilize geodesic distance and Human Physiology characteristic to obtain 5 unique points that human body is positioned at head, two hands, bipod, then from described 5 unique points, the central point of resulting each the piecemeal junction of Connection Step (2) successively, thus the main bone of human body palm, underarm, upper arm, sole, shank, thigh, head, trunk formed.

2. the skeleton extracting method based on visual human's dynamic model according to claim 1 is characterized in that: describedly multi-C vector is carried out cluster adopt the Mean-Shift method.

3. the skeleton extracting method based on visual human's dynamic model according to claim 1, it is characterized in that: the attitude of described reference model is arbitrarily.

Images