CN104268138B - Merge the human body motion capture method of depth map and threedimensional model - Google Patents
Merge the human body motion capture method of depth map and threedimensional model Download PDFInfo
- Publication number
- CN104268138B CN104268138B CN201410205213.8A CN201410205213A CN104268138B CN 104268138 B CN104268138 B CN 104268138B CN 201410205213 A CN201410205213 A CN 201410205213A CN 104268138 B CN104268138 B CN 104268138B
- Authority
- CN
- China
- Prior art keywords
- human action
- action
- human
- depth
- depth information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/285—Analysis of motion using a sequence of stereo image pairs
Abstract
The present invention relates to a kind of human body motion capture method for merging depth map and threedimensional model.Optical motion catching method needs binding mark point, and mutual inconvenience, mark point is easily obscured, blocked.The depth information of present invention collection human action, remove moving target background, obtain complete human action depth information, human body three-dimensional point cloud information is changed into, human action threedimensional model is obtained, sets up database, corresponded with the human action skeleton database data of foundation, the depth information for extracting human action to be identified builds threedimensional model, and then carrying out similitude with the human action in three-dimensional modeling data storehouse matches, and capturing movement result is used as by similitude sequence output human action skeleton.The present invention is without the install sensor on human body or addition mark point, it is convenient easily to realize, the method using canonical Time alignment is matched in motion sequence, the precision of two sequences match is improved, the time of matching is set significantly to drop, it is ensured that the speed and precision of capturing movement.
Description
Technical field
The invention belongs to technical field of multimedia information retrieval, and in particular to a kind of people of fusion depth map and threedimensional model
Body motion capture method.
Background technology
Human motion capture technology is the hot issue in multimedia information retrieval field, particularly in video display animation, game
Deng development in, with wide application prospect, domestic and international many research institutions are just being directed to the research in this direction.In recent years,
With the fast development of capturing movement technology, the rise in the field such as three-dimensional video display animation, game, people's interaction of new generation is very more
Complicated and true to nature human action needs quickly captured application, it is necessary to which a kind of fast and effectively method carries out human motion
Capture.The method for capturing movement for the optical profile type having pointed out at present, is based primarily upon principle of computer vision, by specific in target
The monitoring and tracking of luminous point completes motion-captured task.But above-mentioned method for capturing movement has some not enough：
(1) capturing movement of optical profile type needs the binding mark point with performing artist when in use, and needs performing artist to wear
Upper special performance clothes, interaction is got up and inconvenient.When complicated movement, the mark points of different parts is likely to occur mixed
Confuse, block, produce error result, at this moment need manual intervention last handling process.
(2) although it can catch real time kinematics, post processing (including the identification of mark point, tracking, the meter of space coordinate
Calculate) workload it is larger, illumination, reflection case for performance venue have certain requirement, and device calibration is also more loaded down with trivial details.
The content of the invention
, can be effective gram it is an object of the invention to provide a kind of human body motion capture method for merging depth map and threedimensional model
Taking movement range in existing method for capturing movement is restricted, catches effect distortion factor height and the larger technological deficiency of error.
The technical solution adopted in the present invention is：
Merge the human body motion capture method of depth map and threedimensional model, it is characterised in that：
Realized by following steps：
Step one：The depth information of human action is gathered, moving target background is removed, complete human action depth is obtained
Information；
Step 2：The human action depth information information of extraction is converted to the three-dimensional point cloud information of adult body, human body is entered
Row three-dimensional reconstruction, obtains the threedimensional model of human action；
Step 3：Repeat step one and step 2, based on the human action depth information largely gathered, set up human action
Three-dimensional modeling data storehouse M={ Y1,Y2..., Yn, M represents three-dimensional modeling data storehouse, YnRepresent n-th of threedimensional model；
Step 4：Human action skeleton is constructed according to the threedimensional model of organization of human body and human action, human body is set up and moves
The skeleton data storehouse G={ S of work1,S2,…Sn, G represents human skeleton database, wherein SnN-th of human skeleton data is represented,
Wherein the skeleton data of human action is corresponded with three-dimensional modeling data；
Step 5：The depth information of human action to be identified is extracted, human action to be identified is built based on depth information
Threedimensional model, then carries out similitude with the human action in three-dimensional modeling data storehouse and matches, by similitude sequence output human body
Skeleton is acted, the minimum skeleton of similarity distance value is used as capturing movement result as optimal skeleton using optimal skeleton；
In step one, the human action depth information based on collection removes moving target background, obtains complete human body and moves
Make concretely comprising the following steps for depth information：
(1) human action depth map F (x, y, a d of collectionp) represent, wherein, x,yRespectively under pixel coordinate system
Abscissa and ordinate, dpFor depth information；Assuming that being based on the threshold value that depth information splits background area and target area：
T=(maxDepthVulue+minDepthVulue)/2
Wherein, maxDepthVulue and minDepthVulue are respectively the maximum and minimum value of image depth values, and T is remembered
Record is in T0In, according to threshold value T, by F (x, y, dp) it is divided into background area and target area, background area and target area are obtained respectively
The average depth value du in domain1And du2；
(2) T=(du are recalculated1+du2)/2, judge T and T0It is whether equal, if unequal, then by T records in T0
In, repeat the above steps, until T=T0Set up, termination algorithm；Using the T finally obtained as optimal threshold to F (x, y, dp) enter
Row segmentation, removes background, obtains complete human action depth information d0；
In step 2, the human action depth information of extraction is converted to the three-dimensional point cloud information of adult body, human body is carried out
Three-dimensional reconstruction, the threedimensional model for obtaining human action is concretely comprised the following steps：
(1) by the depth information d extracted0Normalization, it is assumed that the number of depth value is N, is calculated in depth value most
Big value max d0(k) with minimum value min d0(k), the depth value after normalization is：
Z (n)=(d0(n)-min d0(k))/(max d0(k)-min d0(k)) k, n=1,2 ..., N
(2) Z=z (n) is made, world coordinate system is that, using video camera as origin, video camera can be regarded as after demarcation
Preferable imaging model, according to simple similar triangles shift theory：
The X for obtaining world coordinate system, Y value can therefrom be calculated, wherein, x, y be respectively pixel coordinate system abscissa and
Ordinate, f is the focal length of video camera；The three-dimensional point cloud information (X, Y, Z) of human action may finally be obtained；
In step 4, the specific steps of human action skeleton are constructed according to the threedimensional model of organization of human body and human action
For：
(1) body trunk is approximately considered as quadrangle in depth map, is designated as Q；
Two summits above Q are exactly shoulder joint node a1And a2Position；
Neck joint point b is a1、a2The midpoint of line；
Along b points upwards, it is exactly topmost head node, c position；
Two summits below Q are hip joint d1And d2Position；
E is in d for hip artis1、d2Point midway；
(2) determine hand point and elbow joint position from a1、a2Place proceeds by search, if arm is stretched, and is set about according to upper
Arm lengths ratio-dependent elbow joint point f1、f2With hand point g1、g2Position；If arm bending, have in the position of elbow joint point
Flex point, the position of flex point is f1、f2, along f1、f2Continue search for being hand point g to terminal1、g2Position；
Similarly, knee joint point h can be determined according to hand point and elbow joint method for determining position1、h2And foot joint
Point i1、i2Position；
(3) each body joint point coordinate is connected by organization of human body order with straight line, has just obtained human action skeleton；
In step 5, the depth information of human action to be identified is extracted, human action to be identified is built based on depth information
Threedimensional model, then carry out concretely comprising the following steps of matching of similitude with the human action in three-dimensional modeling data storehouse：
(1) human action that threedimensional model is represented is clustered with hierarchical clustering algorithm：
1. assume that a parameter m represents the number of final clustering cluster, regard the data of each input as a single number
According to cluster D0, obtain the nearest cluster D adjacent with each Dx, the distance between aggregate of data can regard the distance between aggregate of data center as；
2. by two closest aggregate of data DpAnd DqMerge, the new aggregate of data D of generationn, then calculate DnWith it
The distance of its cluster, if the number of current data cluster is more than m, then branch to the merging for 2. proceeding aggregate of data, otherwise calculate
Method terminates, and obtains cluster result D={ u1,u2,…,um, wherein u represents the value of each class；
(2) each action action frame of human action sequence to be identified is calculated each with human action sequence in three-dimensional modeling data storehouse
The distance between individual action action frame：
1. assume that any one threedimensional model cluster result is expressed as in human action to be identified and database：
Di={ u '1, u '2..., u 'm}
Dj={ v1, v2..., vm}
Since the root node of clustering tree, depth-first search is carried out until leaf node；
2. to D during traversal searchiAnd DjIn each class calculation and object distance, the distance of each class is summed,
Obtain the Euclidean distance of two clustering trees：
(3) Optimum Matching path is calculated with the method for canonical Time alignment, obtains Optimum Matching sequence：
1. assume that threedimensional model action sequence to be identified isThree-dimensional modeling data storehouse
In set sequence beM and n represent the length of two action sequences, two respectively
The distance between action action frame of each in sequence can calculate obtain in aforementioned manners；
2. because m and n may be unequal, its corresponding matching relationship may have many kinds, be advised using the canonical time
Whole method, calculates the minimum range J between sequence, while obtaining best matching path PxAnd Py：
Here, Wx=W (px)∈{0,1}m×lAnd Wy=W (py)∈{0,1}n×lIt is two binary regular matrixes, to every
The t ∈ { 1 of one step:L } haveDuring other situationsWherein l is the need that canonical time wrapping algorithm is automatically selected
The number for the action action frame to be matched, l >=max (m, n)；φ () is regular terms：
PxAnd PyAll possible coupling path is represented, it is necessary to which the constraints met is：
Ψ={ { Px,Py}|Px∈(1:m)l,Py∈(1:n)l
Border：
Monotonicity：
Continuity：
WithIt is linear change matrix, d≤min (dx,dy), the constraints that need to be met
For：
Wherein, λ ∈ [0,1]；
3. above step is repeated, using canonical Time alignment method, is calculated in three-dimensional modeling data storehouse per set sequence
Arrange the minimum range J and best matching path P of Y respectively with XxAnd Py；Then each J minimum value P corresponding with its is obtainedxWith
Py, so as to obtain corresponding best match sequence.
The present invention has advantages below：
(1) method involved in the present invention uses relevant device sampling depth information, and performing artist on human body without installing
Sensor or addition mark point, movable amplitude are big, few for performance venue requirement, are convenient and easy to realize.
(2) present invention is mainly based upon video method for the capturing movement technology of conventional unmarked point, shoots two dimension
Image sequence reaches the purpose of capturing movement, and the present invention has merged depth information, solves in the X-Y scheme without depth information
Body blocks the problem of causing body part loss of learning certainly as in, it is ensured that the integrality and reliability of acquisition of information.
(3) present invention uses hierarchical clustering method, and the similarity of distance and rule is easily defined between aggregate of data, by data volume compared with
Big three dimensional point cloud is reduced into several classes, in order to which later use Depth Priority Searching calculates two frame threedimensional models
Euclidean distance, reduces amount of calculation.
(4) present invention is matched using canonical Time alignment (Canonical Time Warping, CTW) in motion sequence
Method, this method has the advantages that algorithm robust, even and if test mode sequence and reference sequences pattern in database
Time scale can not be completely the same, and it remains to preferably complete the pattern match between cycle tests and reference sequences.CTW is base
In the method for Dynamic Programming, the 2. middle formula of step 5 (3) step is added with just in the object function of CTW methods, object function
Then item and two regular matrixes of binary system, the possibility of unique solution is provided for object function, improves two sequences match
Precision, makes the time of matching significantly drop, it is ensured that the speed and precision of capturing movement.
Brief description of the drawings
Fig. 1 is the overview flow chart of the inventive method.
Fig. 2 is the idiographic flow block diagram of step one of the present invention.
Fig. 3 is the idiographic flow block diagram of step 5 of the present invention.
Embodiment
With reference to embodiment, the present invention will be described in detail.
Fusion depth map involved in the present invention and the human body motion capture method of threedimensional model, are realized by following steps：
Step one：The depth information of human action is gathered using relevant device, moving target background is removed, obtained completely
Human action depth information.
Human action depth information based on collection, removes moving target background, obtains complete human action depth letter
What is ceased concretely comprises the following steps：
(1) human action depth map F (x, y, a d of collectionp) represent, wherein, under x, y are respectively pixel coordinate system
Abscissa and ordinate, dpFor depth information；Assuming that being based on the threshold value that depth information splits background area and target area：
T=(maxDepthVulue+minDepthVulue)/2
Wherein, maxDepthVulue and minDepthVulue are respectively the maximum and minimum value of image depth values, and T is remembered
Record is in T0In, according to threshold value T, by F (x, y, dp) it is divided into background area and target area, background area and target area are obtained respectively
The average depth value du in domain1And du2；
(2) T=(du are recalculated1+du2)/2, judge T and T0It is whether equal, if unequal, then by T records in T0
In, repeat the above steps, until T=T0Set up, termination algorithm；Using the T finally obtained as optimal threshold to F (x, y, dp) enter
Row segmentation, removes background, obtains complete human action depth information d0。
Step 2：The human action depth information of extraction is converted to the three-dimensional point cloud information of adult body, three are carried out to human body
Dimension is rebuild, and is obtained the threedimensional model of human action, is concretely comprised the following steps：
(1) by the depth information d extracted0Normalization, it is assumed that the number of depth value is N, is calculated in depth value most
Big value maximum max d0(k) with minimum value min d0(k), the depth value after normalization is：
Z (n)=(d0(n)-mind0(k))/(maxd0(k)-mind0(k)) k, n=1,2 ..., N
(2) Z=z (n) is made, world coordinate system is that, using video camera as origin, video camera can be regarded as after demarcation
Preferable imaging model, according to simple similar triangles shift theory：
The X for obtaining world coordinate system, Y value can therefrom be calculated, wherein, x, y be respectively pixel coordinate system abscissa and
Ordinate, f is the focal length of video camera.The three-dimensional point cloud information (X, Y, Z) of human action may finally be obtained.
Step 3：Repeat step one and step 2, based on the human action depth information as much as possible largely gathered, build
Three-dimensional modeling data storehouse M={ the Y of vertical human action1,Y2..., Yn, M represents three-dimensional modeling data storehouse, YnRepresent n-th of three-dimensional
Model.
Step 4：Human action skeleton is constructed according to the threedimensional model of organization of human body and human action, human body is set up and moves
The skeleton data storehouse G={ S of work1,S2,…Sn, G represents human skeleton database, wherein SnN-th of human skeleton data is represented,
Wherein the skeleton data of human action is corresponded with three-dimensional modeling data.
Concretely comprising the following steps for human action skeleton is constructed according to the threedimensional model of organization of human body and human action：
(1) body trunk is approximately considered as quadrangle in depth map, is designated as Q；
Two summits above Q are exactly shoulder joint node a1And a2Position；
Neck joint point b is a1、a2The midpoint of line；
Along b points upwards, it is exactly topmost head node, c position；
Two summits below Q are hip joint d1And d2Position；
E is in d for hip artis1、d2Point midway；
(2) determine hand point and elbow joint position from a1、a2Place proceeds by search, if arm is stretched, and is set about according to upper
Arm lengths ratio-dependent elbow joint point f1、f2With hand point g1、g2Position；If arm bending, have in the position of elbow joint point
Flex point, the position of flex point is f1、f2, along f1、f2Continue search for being hand point g to terminal1、g2Position；
Similarly, knee joint point h can be determined according to hand point and elbow joint method for determining position1、h2And foot joint
Point i1、i2Position；
(3) each body joint point coordinate is connected by organization of human body order with straight line, has just obtained human action skeleton.
Step 5：The depth information of human action to be identified is extracted, human action to be identified is built based on depth information
Threedimensional model, then carries out similitude with the human action in three-dimensional modeling data storehouse and matches, by similitude sequence output human body
Skeleton is acted, the minimum skeleton of similarity distance value is used as capturing movement result as optimal skeleton using optimal skeleton.
The depth information of human action to be identified is extracted, the three-dimensional mould of human action to be identified is built based on depth information
Type, then with concretely comprising the following steps that the human action progress similitude in three-dimensional modeling data storehouse is matched：
(1) human action for representing threedimensional model is clustered with hierarchical clustering algorithm (THA)：
1. assume that a parameter m represents the number of final clustering cluster, regard the data of each input as a single number
According to cluster D0, obtain the nearest cluster D adjacent with each Dx, the distance between aggregate of data can regard the distance between aggregate of data center as；
2. by two closest aggregate of data DpAnd DqMerge, the new aggregate of data D of generationn, then calculate DnWith it
The distance of its cluster, if the number of current data cluster is more than m, then branch to the merging for 2. proceeding aggregate of data, otherwise calculate
Method terminates, and obtains cluster result D={ u1,u2,…,um, wherein u represents the value of each class；
(2) each action action frame of human action sequence to be identified is calculated each with human action sequence in three-dimensional modeling data storehouse
The distance between individual action action frame：
1. assume that any one threedimensional model cluster result is expressed as in human action to be identified and database：
Di={ u '1, u '2..., u 'm}
Dj={ v1, v2..., vm}
Since the root node of clustering tree, depth-first search is carried out until leaf node；
2. to D during traversal searchiAnd DjIn each class calculation and object distance, the distance of each class is summed,
Obtain the Euclidean distance of two clustering trees：
(3) Optimum Matching road is calculated with the method for canonical Time alignment (Canonical Time Warping, CTW)
Footpath, obtains Optimum Matching sequence：
1. assume that threedimensional model action sequence to be identified isThree-dimensional modeling data storehouse
In set sequence beM and n represent the length of two action sequences, two respectively
The distance between action action frame of each in sequence can calculate obtain in aforementioned manners；
2. because m and n may be unequal, its corresponding matching relationship may have many kinds, be advised using the canonical time
Whole method, calculates the minimum range J between sequence, while obtaining best matching path PxAnd Py：
Here, Wx=W (px)∈{0,1}m×lAnd Wy=W (py)∈{0,1}n×lIt is two binary regular matrixes, to every
The t ∈ { 1 of one step:L } haveDuring other situationsWherein l is the need that canonical time wrapping algorithm is automatically selected
The number for the action action frame to be matched, l >=max (m, n).φ () is regular terms：
PxAnd PyAll possible coupling path is represented, it is necessary to which the constraints met is：
Ψ={ { Px,Py}|Px∈(1:m)l,Py∈(1:n)l
Border：
Monotonicity：
Continuity：
WithIt is linear change matrix, the pact that need to be met
Beam condition is：
Wherein, λ ∈ [0,1]；
3. above step is repeated, using canonical Time alignment method, is calculated in three-dimensional modeling data storehouse per set sequence
Arrange the minimum range J and best matching path P of Y respectively with XxAnd Py.Then each J minimum value P corresponding with its is obtainedxWith
Py, so as to obtain corresponding best match sequence.
Present disclosure is not limited to cited by embodiment, and those of ordinary skill in the art are by reading description of the invention
And any equivalent conversion taken technical solution of the present invention, it is that claim of the invention is covered.
Claims (1)
1. merge the human body motion capture method of depth map and threedimensional model, it is characterised in that：
Realized by following steps：
Step one：The depth information of human action is gathered, moving target background is removed, complete human action depth letter is obtained
Breath；
Step 2：The human action depth information information of extraction is converted to the three-dimensional point cloud information of adult body, three are carried out to human body
Dimension is rebuild, and obtains the threedimensional model of human action；
Step 3：Repeat step one and step 2, based on the human action depth information largely gathered, set up the three of human action
Dimension module database M={ Y1,Y2..., Yn, M represents three-dimensional modeling data storehouse, YnRepresent n-th of threedimensional model；
Step 4：Human action skeleton is constructed according to the threedimensional model of organization of human body and human action, human action is set up
Skeleton data storehouse G={ S1,S2,…Sn, G represents human skeleton database, wherein SnN-th of human skeleton data is represented, wherein
The skeleton data of human action is corresponded with three-dimensional modeling data；
Step 5：The depth information of human action to be identified is extracted, the three-dimensional of human action to be identified is built based on depth information
Model, then carries out similitude with the human action in three-dimensional modeling data storehouse and matches, by similitude sequence output human action
Skeleton, the minimum skeleton of similarity distance value is used as capturing movement result as optimal skeleton using optimal skeleton；
In step one, the human action depth information based on collection removes moving target background, obtains complete human action deep
Spend concretely comprising the following steps for information：
(1) human action depth map F (x, y, a d of collectionp) represent, wherein, x, y are respectively the horizontal stroke under pixel coordinate system
Coordinate and ordinate, dpFor depth information；Assuming that being based on the threshold value that depth information splits background area and target area：
T=(maxDepthVulue+minDepthVulue)/2
Wherein, maxDepthVulue and minDepthVulue are respectively the maximum and minimum value of image depth values, and T records are existed
T0In, according to threshold value T, by F (x, y, dp) it is divided into background area and target area, background area and target area are obtained respectively
Average depth value du1And du2；
(2) T=(du are recalculated1+du2)/2, judge T and T0It is whether equal, if unequal, then by T records in T0In, weight
Multiple above-mentioned steps, until T=T0Set up, termination algorithm；Using the T finally obtained as optimal threshold to F (x, y, dp) divided
Cut, remove background, obtain complete human action depth information d0；
In step 2, the human action depth information of extraction is converted to the three-dimensional point cloud information of adult body, human body carried out three-dimensional
Rebuild, the threedimensional model for obtaining human action is concretely comprised the following steps：
(1) by the depth information d extracted0Normalization, it is assumed that the number of depth value is N, calculates the maximum in depth value
max d0(k) with minimum value min d0(k), the depth value after normalization is：
Z (n)=(d0(n)-min d0(k))/(max d0(k)-min d0(k)) k, n=1,2 ..., N
(2) Z=z (n) is made, world coordinate system is that, using video camera as origin, video camera can regard preferable as after demarcation
Imaging model, according to simple similar triangles shift theory：
The X for obtaining world coordinate system, Y value can be therefrom calculated, wherein, x, y are respectively the abscissa and vertical seat of pixel coordinate system
Mark, f is the focal length of video camera；The three-dimensional point cloud information (X, Y, Z) of human action may finally be obtained；
In step 4, concretely comprising the following steps for human action skeleton is constructed according to the threedimensional model of organization of human body and human action：
(1) body trunk is approximately considered as quadrangle in depth map, is designated as Q；
Two summits above Q are exactly shoulder joint node a1And a2Position；
Neck joint point b is a1、a2The midpoint of line；
Along b points upwards, it is exactly topmost head node, c position；
Two summits below Q are hip joint d1And d2Position；
E is in d for hip artis1、d2Point midway；
(2) determine hand point and elbow joint position from a1、a2Place proceeds by search, if arm is stretched, long according to upper lower arms
Spend ratio-dependent elbow joint point f1、f2With hand point g1、g2Position；If arm bending, turned in the position of elbow joint point
Point, the position of flex point is f1、f2, along f1、f2Continue search for being hand point g to terminal1、g2Position；
Similarly, knee joint point h can be determined according to hand point and elbow joint method for determining position1、h2With foot joint point
i1、i2Position；
(3) each body joint point coordinate is connected by organization of human body order with straight line, has just obtained human action skeleton；
In step 5, the depth information of human action to be identified is extracted, the three of human action to be identified are built based on depth information
Dimension module, then with concretely comprising the following steps that the human action progress similitude in three-dimensional modeling data storehouse is matched：
(1) human action that threedimensional model is represented is clustered with hierarchical clustering algorithm：
1. assume that a parameter m represents the number of final clustering cluster, regard the data of each input as a single aggregate of data
D0, obtain the nearest cluster D adjacent with each Dx, the distance between aggregate of data can regard the distance between aggregate of data center as；
2. by two closest aggregate of data DpAnd DqMerge, the new aggregate of data D of generationn, then calculate DnWith other clusters
Distance, if the number of current data cluster be more than m, then branch to the merging for 2. proceeding aggregate of data, otherwise algorithm knot
Beam, obtains cluster result D={ u1,u2,…,um, wherein u represents the value of each class；
(2) calculating each action action frame of human action sequence to be identified and the human action sequence in three-dimensional modeling data storehouse, each is moved
Make the distance between frame：
1. assume that any one threedimensional model cluster result is expressed as in human action to be identified and database：
Di={ u '1, u '2..., u 'm}
Dj={ v1, v2..., vm}
Since the root node of clustering tree, depth-first search is carried out until leaf node；
2. to D during traversal searchiAnd DjIn each class calculation and object distance, the distance of each class is summed, obtained
The Euclidean distance of two clustering trees：
(3) Optimum Matching path is calculated with the method for canonical Time alignment, obtains Optimum Matching sequence：
1. assume that threedimensional model action sequence to be identified isIn three-dimensional modeling data storehouse
Set sequence isM and n represent the length of two action sequences, two sequences respectively
In the distance between each action action frame can calculate obtain in aforementioned manners；
2. because m and n may be unequal, its corresponding matching relationship may have many kinds, utilize canonical Time alignment
Method, calculates the minimum range J between sequence, while obtaining best matching path PxAnd Py：
Here, Wx=W (px)∈{0,1}m×lAnd Wy=W (py)∈{0,1}n×lIt is two binary regular matrixes, to each step
T ∈ { 1:L } there is wpt,t=1, w during other situationspt,t=0, wherein l are the need for canonical time wrapping algorithm is automatically selected
The number for the action action frame matched somebody with somebody, l >=max (m, n)；φ () is regular terms：
PxAnd PyAll possible coupling path is represented, it is necessary to which the constraints met is：
Ψ={ { Px,Py}|Px∈(1:m)l,Py∈(1:n)l
Border：
Monotonicity：
Continuity：
WithIt is linear change matrix, d≤min (dx,dy), the constraints that need to be met is：
Wherein, λ ∈ [0,1]；
3. above step is repeated, using canonical Time alignment method, each group of Y points of action sequence in three-dimensional modeling data storehouse is calculated
Not with X minimum range J and best matching path PxAnd Py；Then each J minimum value P corresponding with its is obtainedxAnd Py, from
And obtain corresponding best match sequence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410205213.8A CN104268138B (en) | 2014-05-15 | 2014-05-15 | Merge the human body motion capture method of depth map and threedimensional model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410205213.8A CN104268138B (en) | 2014-05-15 | 2014-05-15 | Merge the human body motion capture method of depth map and threedimensional model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104268138A CN104268138A (en) | 2015-01-07 |
| CN104268138B true CN104268138B (en) | 2017-08-15 |
Family
ID=52159660
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410205213.8A Expired - Fee Related CN104268138B (en) | 2014-05-15 | 2014-05-15 | Merge the human body motion capture method of depth map and threedimensional model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104268138B (en) |
Families Citing this family (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104700452B (en) * | 2015-03-24 | 2016-03-02 | 中国人民解放军国防科学技术大学 | A kind of 3 D human body attitude mode matching process towards any attitude |
| CN104680582B (en) * | 2015-03-24 | 2016-02-24 | 中国人民解放军国防科学技术大学 | A kind of three-dimensional (3 D) manikin creation method of object-oriented customization |
| CN105069423B (en) * | 2015-07-29 | 2018-11-09 | 北京格灵深瞳信息技术有限公司 | A kind of human body attitude detection method and device |
| CN105335722B (en) * | 2015-10-30 | 2021-02-02 | 商汤集团有限公司 | Detection system and method based on depth image information |
| JP6596309B2 (en) * | 2015-11-11 | 2019-10-23 | 株式会社東芝 | Analysis apparatus and analysis method |
| CN105930773A (en) * | 2016-04-13 | 2016-09-07 | 中国农业大学 | Motion identification method and device |
| CN106441275A (en) * | 2016-09-23 | 2017-02-22 | 深圳大学 | Method and device for updating planned path of robot |
| CN106599806A (en) * | 2016-12-01 | 2017-04-26 | 西安理工大学 | Local curved-surface geometric feature-based human body action recognition method |
| US20180295338A1 (en) * | 2017-04-10 | 2018-10-11 | Eys3D Microelectronics, Co. | Depth processing system capable of capturing depth information from multiple viewing points |
| CN106998430B (en) * | 2017-04-28 | 2020-07-21 | 北京瑞盖科技股份有限公司 | Multi-camera-based 360-degree video playback method |
| CN107212975A (en) * | 2017-07-17 | 2017-09-29 | 徐彬 | Wheelchair and method with intelligent rescue function |
| CN107551551B (en) * | 2017-08-09 | 2021-03-26 | Oppo广东移动通信有限公司 | Game effect construction method and device |
| CN108096836B (en) * | 2017-12-20 | 2021-05-04 | 深圳市百恩互动娱乐有限公司 | Method for making game by real-person real shooting |
| CN108121963B (en) * | 2017-12-21 | 2021-08-24 | 北京奇虎科技有限公司 | Video data processing method and device and computing equipment |
| CN108510577B (en) * | 2018-01-31 | 2021-03-23 | 中国科学院软件研究所 | Realistic motion migration and generation method and system based on existing motion data |
| CN108392207B (en) * | 2018-02-09 | 2020-12-11 | 西北大学 | Gesture tag-based action recognition method |
| CN108563329B (en) * | 2018-03-23 | 2021-04-27 | 上海数迹智能科技有限公司 | Human body arm position parameter extraction algorithm based on depth map |
| CN109215128B (en) * | 2018-08-09 | 2019-12-24 | 北京华捷艾米科技有限公司 | Object motion attitude image synthesis method and system |
| CN110020611B (en) * | 2019-03-17 | 2020-12-08 | 浙江大学 | Multi-person motion capture method based on three-dimensional hypothesis space clustering |
| CN110276266B (en) * | 2019-05-28 | 2021-09-10 | 暗物智能科技(广州)有限公司 | Rotation-based point cloud data processing method and device and terminal equipment |
| CN111354075A (en) * | 2020-02-27 | 2020-06-30 | 青岛联合创智科技有限公司 | Foreground reduction interference extraction method in three-dimensional reconstruction |
| CN112330815A (en) * | 2020-11-26 | 2021-02-05 | 北京百度网讯科技有限公司 | Three-dimensional point cloud data processing method, device and equipment based on obstacle fusion |
| CN117769713A (en) * | 2021-09-14 | 2024-03-26 | 日本电气株式会社 | Method and apparatus for determining abnormal behavior during a period |
| CN114694263B (en) * | 2022-05-30 | 2022-09-02 | 深圳智华科技发展有限公司 | Action recognition method, device, equipment and storage medium |
| CN116385663B (en) * | 2023-05-26 | 2023-08-29 | 北京七维视觉传媒科技有限公司 | Action data generation method and device, electronic equipment and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103294996A (en) * | 2013-05-09 | 2013-09-11 | 电子科技大学 | 3D gesture recognition method |
-
2014
- 2014-05-15 CN CN201410205213.8A patent/CN104268138B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103294996A (en) * | 2013-05-09 | 2013-09-11 | 电子科技大学 | 3D gesture recognition method |
Non-Patent Citations (4)
| Title |
|---|
| canonical time warping for alignment of human behavior;Feng zhou等;《Carnegie mellon university》;20091231;全文 * |
| Human-like character animation of maize driven by motion capture data;wang xue等;《information and computational science》;20111231;全文 * |
| 动态深度数据匹配及其应用;陈炉军;《中国优秀硕士学位论文全文数据库 信息科技辑》;20140415;正文第10-11,14页，20-21页，图2-3，图2-8 * |
| 基于Kinect传感器的骨骼定位研究;罗鸣;《中国优秀硕士学位论文全文数据库 信息科技辑》;20140415;正文第10页，图2.5 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104268138A (en) | 2015-01-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104268138B (en) | Merge the human body motion capture method of depth map and threedimensional model | |
| CN102682302B (en) | Human body posture identification method based on multi-characteristic fusion of key frame | |
| Schonberger et al. | Structure-from-motion revisited | |
| CN102855470B (en) | Estimation method of human posture based on depth image | |
| CN108537191B (en) | Three-dimensional face recognition method based on structured light camera | |
| JP6025845B2 (en) | Object posture search apparatus and method | |
| CN108154104B (en) | Human body posture estimation method based on depth image super-pixel combined features | |
| WO2015149302A1 (en) | Method for rebuilding tree model on the basis of point cloud and data driving | |
| CN105930770B (en) | A kind of human motion recognition method based on Gaussian process latent variable model | |
| Ran et al. | Applications of a simple characterization of human gait in surveillance | |
| Zhang et al. | Data-driven synthetic modeling of trees | |
| CN105931283B (en) | A kind of 3-dimensional digital content intelligence production cloud platform based on motion capture big data | |
| WO2022213612A1 (en) | Non-contact three-dimensional human body size measurement method | |
| CN108305283A (en) | Human bodys' response method and device based on depth camera and basic form | |
| CN102467753A (en) | Method and system for reconstructing time-varying point cloud based on framework registration | |
| Uddin et al. | Human Activity Recognition via 3-D joint angle features and Hidden Markov models | |
| WO2017079918A1 (en) | Indoor scene scanning reconstruction method and apparatus | |
| Ma et al. | Automatic branch detection of jujube trees based on 3D reconstruction for dormant pruning using the deep learning-based method | |
| CN109101864A (en) | The upper half of human body action identification method returned based on key frame and random forest | |
| CN108648194A (en) | Based on the segmentation of CAD model Three-dimensional target recognition and pose measuring method and device | |
| CN109766873A (en) | A kind of pedestrian mixing deformable convolution recognition methods again | |
| Zhao et al. | Character‐object interaction retrieval using the interaction bisector surface | |
| Song et al. | Attention-oriented action recognition for real-time human-robot interaction | |
| Wang et al. | Action recognition using edge trajectories and motion acceleration descriptor | |
| CN114689038A (en) | Fruit detection positioning and orchard map construction method based on machine vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20170815 Termination date: 20190515 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |