CN109614899A - A kind of human motion recognition method based on Lie group feature and convolutional neural networks - Google Patents

A kind of human motion recognition method based on Lie group feature and convolutional neural networks Download PDF

Info

Publication number
CN109614899A
CN109614899A CN201811446456.5A CN201811446456A CN109614899A CN 109614899 A CN109614899 A CN 109614899A CN 201811446456 A CN201811446456 A CN 201811446456A CN 109614899 A CN109614899 A CN 109614899A
Authority
CN
China
Prior art keywords
limbs
lie group
lie
feature
convolutional neural
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811446456.5A
Other languages
Chinese (zh)
Other versions
CN109614899B (en
Inventor
蔡林沁
丁和恩
陆相羽
隆涛
陈思维
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zhifeng Technology Co.,Ltd.
Original Assignee
Chongqing University of Post and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University of Post and Telecommunications filed Critical Chongqing University of Post and Telecommunications
Priority to CN201811446456.5A priority Critical patent/CN109614899B/en
Publication of CN109614899A publication Critical patent/CN109614899A/en
Application granted granted Critical
Publication of CN109614899B publication Critical patent/CN109614899B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/29—Graphical models, e.g. Bayesian networks
    • G06F18/295—Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The human motion recognition method based on Lie group feature and convolutional neural networks that the present invention relates to a kind of, belongs to CRT technology field.This method comprises: S1: data acquisition extracts skeleton information using Microsoft somatosensory device Kinect, obtains the motion information of experimenter；S2: Lie group feature is extracted, take a kind of Lie group bone representation method that the relative dimensional geometrical relationship between each limbs of human body is simulated using rigid limbs transformation, human action is modeled as a series of curves in Lie group, it will be the curve based on Lie algebra space based on the curve mapping in Lie group space using logarithmic mapping and then in conjunction with the corresponding relationship between Lie group and Lie algebra；S3: tagsort merges Lie group feature and convolutional neural networks and allows convolutional neural networks to be learnt to Lie group feature, be classified using Lie group feature training convolutional neural networks, to realize that human action identifies.The present invention can obtain good recognition effect.

Description

A kind of human motion recognition method based on Lie group feature and convolutional neural networks
Technical field
The invention belongs to CRT technology fields, are related to a kind of human body based on Lie group feature and convolutional neural networks Action identification method.
Background technique
With the fast development of science and technology, more natural human-computer interaction becomes the more more and more urgent needs of people, Ren Mengeng Add serious hope computer that can think deeply and understand the signal of extraneous input as human brain, understands the daily behavioral activity of the mankind, with Convenient for more easily naturally being exchanged with computer.
Human action identification refers to digital picture or video signal flow etc. for object, passes through image procossing and automatic identification The methods of, obtain a kind of practical technique of human action information.Due to the variability of human action, camera motion, light intensity The presence of the problems such as otherness under the conditions of variation, the gap of different people figure, human body varying environment so that human action identification Research become a multi-crossed disciplines and extremely challenging technical problem.
In recent years, since human action identifies that, in computer vision, human-computer interaction, video monitoring, health care virtually shows The extensive use in the fields such as real, already becomes the research field when next hot topic, by computer vision, artificial intelligence etc. The favor of area research person.Currently, the method for most human body action recognitions mainly uses manual extraction feature.This method master It is divided into feature detection and feature describes two stages, wherein common characteristic detection method is if any 3D Corner Detection, Cuboid Algorithm and Hessian3D matrix；And common attribute description son such as Cuboid algorithm, histograms of oriented gradients (HOG), light stream histogram Scheme (HOF), enhanced intensive Trajectory Arithmetic (iDT) etc..But since the method for manual extraction feature is more time-consuming, Er Qieti The quality of feature is taken to be highly dependent on the experience of researcher, so this method based on manual extraction feature is slowly just lost The favour of researcher is gone.Given this reason, Many researchers propose to carry out human body with the color image video of human motion Action recognition, this method achieve certain effect, but since color image video lacks the three-dimensional spatial information of human motion, The description of human motion cannot be accomplished comprehensively, and artificially block, the factors such as illumination variation under the influence of, unavoidably Cause action recognition inaccurate in addition it is unrecognized as a result, having embodied great limitation.
In recent years, with the appearance of some depth transducers, the Kinect to produce such as Microsoft, HuaShuo Co., Ltd produce Xtion PRO etc. greatly changes the method extracted for body motion information.Use depth transducer energy convenient and efficient Body motion information is obtained, compared to color image, depth image and bone information have significantly in description human motion Advantage, on the one hand, depth transducer equipment is not only easy to operate, and greatly simplifies the calibration process of common camera； On the other hand, the depth image obtained directly contains the depth information of human body, can effectively overcome the shadow of illumination variation etc. It rings and depth image has more distinction than the texture and color description of color image for the description of geometry.So base The research interest of numerous researchers is caused in the human action identification of bone information, has emerged in large numbers the achievement of many stages.Closely Nian Lai, many scholars' propositions extract human body motion feature in popular world, pass through the relative dimensional geometry of human body difference limbs Relationship can more fully describe the feature of athletic performance, relative to the joint being only connected with each other between limbs The more advantage such as angle change between point change in location, limbs.
In the classification of motion, the method for some deep learnings proposed in recent years identifies field in image recognition and physical activity Deng successful application, cause extensive concern.Such as convolutional neural networks, depth confidence network are handled to high dimensional data, Feature learning etc. has embodied advantage, for reducing calculation amount, reduces the complexity of identification process, enhances accuracy of identification There is preferable effect.
Therefore, in order to overcome the shortcomings of that traditional-handwork extracts feature, the three-dimensional of the bone information of human motion is made full use of The advantage of spatial information and deep learning, the present invention propose a kind of based on the knowledge of the human action of Lie group feature and convolutional neural networks Other method.
Summary of the invention
In view of this, the purpose of the present invention is to provide a kind of human action based on Lie group feature and convolutional neural networks Recognition methods, this method greatly overcome interference of the traditional technology to extraneous environmental change and human somatotype variation etc., can Preferably overcome some action identification methods based on traditional theorem in Euclid space that can not simulate, state the space complexity of human action And the defect of geometrical relationship；Simultaneously this method can preferably processing movement between Similarity Problem and class between high variability problem； On calculating cost and recognition effect, feature is handled with convolutional neural networks feature can be learnt very well, Classification, also can largely reduce calculating cost；Recognition accuracy is high.
In order to achieve the above objectives, the invention provides the following technical scheme:
A kind of human motion recognition method based on Lie group feature and convolutional neural networks, specifically includes the following steps:
S1: data acquisition extracts skeleton information using Microsoft somatosensory device Kinect, obtains the movement letter of experimenter Breath；
S2: extracting Lie group feature, take it is a kind of using the rigid limbs transformation rotation of such as three-dimensional space (, translate) come mould The Lie group bone representation method of relative dimensional geometrical relationship between anthropomorphic each limbs of body, is modeled as a system in Lie group for human action Curve mapping based on Lie group is base using logarithmic mapping by column curve, and then combine the corresponding relationship between Lie group and Lie algebra Curve in Lie algebra space；
S3: tagsort merges Lie group feature and convolutional neural networks, using Lie group feature training convolutional neural networks, Convolutional neural networks are allowed to be learnt to Lie group feature, be classified, to realize that human action identifies.
Further, it in the step S1, for the skeleton information of acquisition, is normalized, guarantees skeleton ruler Very little equal consistency.
Further, the step S2 is specifically included:
Human skeleton is indicated with S=(V, E), wherein V={ v1..., vNBe artis set, E={ e1..., eMTable Show the set of rigid limbs, wherein N is artis quantity, and M is rigid limbs quantity；Define en1∈R3And en2∈R3It respectively indicates Limbs enBeginning and end；Given a pair of joint limbs emAnd en, the static state of human body, which acts, can use emAnd enBetween opposite geometry Relationship is described, and this description method is summarized are as follows: in local coordinate system will wherein a limbs rotated, move to separately In one limbs same direction same position；Wherein complete rigid limbs conversion process an are as follows: limbs are first around axisWith certain angle Degree θ rotated, rotate to another limbs it is equidirectional after stop rotating, then translate againDistance be allowed to and another limbs It is overlapped.
Further, in the step S2, the complete rigid limbs conversion process specifically:
For rigid limbs emAnd en, by enBy rotating, translating, make itself and emIt is overlapped, obtains emSame enBetween one three Tie up transformation relation are as follows:
Wherein,It indicates with limbs emFor starting point, with limbs enFor the rotary shaft vector of terminal；θM, nIndicate limbs emAround AxisIt rotates to and limbs enEquidirectional angle；For postrotational limbs emMove to limbs enDistance vector；
Similarly, by emBy rotating, translating, make itself and enIt is overlapped, obtains emSame enBetween another three-dimension varying relationship Are as follows:
Wherein,It indicates with limbs emFor starting point, with limbs enFor the rotary shaft vector of terminal；θM, nIndicate limbs emAround AxisIt rotates to and limbs enEquidirectional angle；For postrotational limbs emMove to limbs enDistance vector；
Gather the opposite 3D geometrical relationship between all limbs pair, at a certain moment t, a human body skeleton representation is following shape Formula:
S (t)=(T1,2(t), T2,1(t) ..., TM-1, M(t), TM, M-1(t)) (3)
Wherein, M is rigid limbs quantity, and M (M-1) is the total degree of all rigid limbs transformation, TM, M-1(t) limbs are indicated eM-1With limbs eMBetween three-dimension varying relationship；Bone representation more than, the bone sequence for describing human action indicate For the curve of following form:
{ S (t), t ∈ [0, T '] } (4)
Wherein, T ' is total time；
Remember RI, j(t) it is 3D spin matrix, is expressed asSo (3) can be deformed into:
It will indicate that the curve mapping in Lie group space can be obtained to Lie algebra space:
Wherein, vec (g) indicates vector space.
Further, it is specifically included in the step S3:
S31: by the curve obtained in step S2 dynamic time warping (Dynamic Time Warping, DTW) method Regular processing is carried out, problem is become with fix-rate, before from Lie group space reflection to Lie algebra space, is moved for every class by curve Make, all needs first to calculate a standard curve, then by all snaps of all curves to standard curve, and keep its length consistent.
S32: right using Fourier descriptor (Fourier temporal pyramid, FTP) after DTW is handled Curve is described, by one three layers of time pyramid representation of obtained Fourier descriptor, and by each partial-length A quarter is as its low frequency coefficient, the Feature Descriptor entirely acted, the step be advantageous in that can overcome noise, The unfavorable factors such as time deviation enhance robustness；
S33: after completing Lie group feature extraction, motion characteristic is fused in convolutional neural networks be trained, learn, Classification；Using two-dimensional convolution core in convolution process, convolution exports later are as follows:
Wherein, R is activation primitive ReLU, xiFor input feature vector figure, yiTo export characteristic pattern, wI, jJth interlayer is arrived for i-th layer Weight, bjTo set parameter.
The beneficial effects of the present invention are:
(1) present invention is described human motion using skeleton information, overcomes tradition and carries out manual extraction spy The shortcomings that sign, has stronger robustness for the problems such as noise, rate change.The skeleton letter extracted by equipment such as Kinect Breath greatly overcomes the wrong identification as caused by the external world's care change, blocks, dress ornament changes and human somatotype difference etc. The shortcomings that.When human body makes different movements, there is different positions and angular relationship etc. between corresponding joint and bone, this Description more accurate and effective of a little features for human action.
(2) present invention describes human action using Lie group feature, utilizes the relative dimensional geometrical relationship between parts of body Simulate human action.Currently, most action identification methods based on skeleton information be all the explicit feature of extraction comparison such as, Three-dimensional coordinate, joint angles relative changing value and the speed for passing through the feature such as artis that post-processing is crossed of artis, angle Speed, orientation information, bone angle etc..Although these information based on algebraic operation can describe human body row to a certain extent For, but compared to the description based on three-dimensional geometry space, it can show slightly not comprehensive enough.The present invention utilize limbs between in three dimensions Relative geometrical relation, for motion characteristic extract it is more detailed.Meanwhile feature extraction is carried out in popular world, it can not It is limited to only to calculate geometrical relationship between the limbs being connected, relevant geometrical relationship can also be calculated between disconnected limbs, Can be fabulous overcome some action identification methods based on traditional theorem in Euclid space can not simulate, the space of stating human action it is multiple The defect of polygamy and geometrical relationship, and can preferably overcome due between movement Similarity Problem and class between high variability problem etc. The problem of being unfavorable for action recognition caused by factor.
(3) present invention fusion Lie group feature and convolutional neural networks carry out action recognition, in view of the knot of convolutional neural networks Structure feature, on the one hand, the neuron weight on same Feature Mapping face is identical (weight is shared), and network is learned parallel It practises, compared to other neural networks, reduces training parameter, network structure becomes simpler, more adaptable；Another party Face due to convolutional neural networks processing capacity strong for higher-dimension complex data and can carry out the spy of multidimensional input Property, compared to some other sorting algorithms such as HMM, random forest etc., not only calculating speed is fast, reduces calculating cost, is knowing Also tool has great advantage in terms of other effect.
(4) (such as Florence3D, UTKinect-Action, MSRAction-3D are tested in multiple and different databases Deng), preferable recognition effect is obtained, stronger generalization ability has been embodied, is suitable for human action and identifies field.
Detailed description of the invention
In order to keep the purpose of the present invention, technical scheme and beneficial effects clearer, the present invention provides following attached drawing and carries out Illustrate:
Fig. 1 is the overall framework figure of the method for the invention；
Fig. 2 is Lie group bone representation method (three-dimensional geometry rotation, translation between limbs) schematic diagram that the present invention uses；
Relational graph of the Fig. 3 between Lie group and Lie algebra；
The overall framework figure for the convolutional neural networks that Fig. 4 uses for the present invention and the parameter used.
Specific embodiment
Below in conjunction with attached drawing, a preferred embodiment of the present invention will be described in detail.
Fig. 1 is the overall frame of the human motion recognition method of the present invention based on Lie group feature and convolutional neural networks Frame, as shown in Figure 1, recognition methods groundwork of the present invention is to obtain human body by the somatosensory device Kinect that Microsoft produces The bone information of motion sequence simulates people using rigid limbs transformation (rotation, the translation of such as three-dimensional space) with a kind of Human action is modeled as a series of songs in Lie group by the Lie group bone representation method of the relative dimensional geometrical relationship between each limbs of body Line, and then the corresponding relationship between Lie group and Lie algebra is combined, such as Fig. 3 is reflected the curve based on Lie group space using logarithmic mapping It penetrates as the curve based on Lie algebra space.Finally, fusion Lie group feature and convolutional neural networks, utilize Lie group feature training convolutional Neural network allows convolutional neural networks to be learnt to Lie group feature, be classified, to realize that human action identifies.It is specific real It is as described below to apply scheme:
A kind of human motion recognition method based on Lie group feature and convolutional neural networks, specifically includes the following steps:
S1: data acquisition extracts skeleton information using Microsoft somatosensory device Kinect, obtains the movement letter of experimenter Breath；It for the skeleton information of acquisition, is normalized, guarantees the consistency of skeleton size etc..
S2: extracting Lie group feature, take it is a kind of using the rigid limbs transformation rotation of such as three-dimensional space (, translate) come mould The Lie group bone representation method of relative dimensional geometrical relationship between anthropomorphic each limbs of body, is modeled as a system in Lie group for human action Column curve, and then combine the corresponding relationship between Lie group and Lie algebra, using logarithmic mapping by the curve mapping based on Lie group space For the curve based on Lie algebra space.As shown in Fig. 2, specific human action three-dimensional geometry representation method principle is as follows:
Human skeleton is indicated with S=(V, E), wherein V={ v1..., vNBe artis set, E={ e1..., eMTable Show the set of rigid limbs, wherein N is artis quantity, and M is rigid limbs quantity；Define en1∈R3And en2∈R3It respectively indicates Limbs enBeginning and end；Given a pair of joint limbs emAnd en, the static state of human body, which acts, can use emAnd enBetween opposite geometry Relationship is described, and this description method is summarized are as follows: in local coordinate system will wherein a limbs rotated, move to separately In one limbs same direction same position；Wherein complete rigid limbs conversion process an are as follows: limbs are first around axisWith certain angle Degree θ rotated, rotate to another limbs it is equidirectional after stop rotating, then translate againDistance be allowed to and another limbs It is overlapped.It is specifically shown in Fig. 2.
For rigid limbs emAnd en, by enBy rotating, translating, make itself and emIt is overlapped, obtains emSame enBetween one three Tie up transformation relation are as follows:
Wherein,It indicates with limbs emFor starting point, with limbs enFor the rotary shaft vector of terminal；θM, nIndicate limbs emAround AxisIt rotates to and limbs enEquidirectional angle；For postrotational limbs emMove to limbs enDistance vector；
Similarly, by emBy rotating, translating, make itself and enIt is overlapped, obtains emSame enBetween another three-dimension varying relationship Are as follows:
Wherein,It indicates with limbs emFor starting point, with limbs enFor the rotary shaft vector of terminal；θM, nIndicate limbs emAround AxisIt rotates to and limbs enEquidirectional angle；For postrotational limbs emMove to limbs enDistance vector；
Gather the opposite 3D geometrical relationship between all limbs pair, in moment t, a human body skeleton representation is following form:
S (t)=(T1,2(t), T2,1(t) ..., TM-1, M (t), TM, M-1(t)) (3)
Wherein, M is rigid limbs quantity, and M (M-1) is the total degree of all rigid limbs transformation, TM, M-1(t) limbs are indicated eM-1With limbs eMBetween three-dimension varying relationship；Bone representation more than, the bone sequence for describing human action indicate For the curve of following form:
{ S (t), t ∈ [0, T '] } (4)
Wherein, T ' is total time；
Remember RI, j(t) it is 3D spin matrix, is expressed asSo (3) can be deformed into:
It will indicate that the curve mapping in Lie group space can be obtained to Lie algebra space:
Wherein, vec (g) indicates vector space.
S3: tagsort merges Lie group feature and convolutional neural networks, using Lie group feature training convolutional neural networks, Convolutional neural networks are allowed to be learnt to Lie group feature, be classified, to realize that human action identifies.
For the curve obtained in step S2, regular processing is carried out with dynamic time warping (DTW) method, is become with fix-rate Problem.It is acted for every class, all needs first to calculate a standard curve, then by all snaps of all curves to standard curve, and make Its length is consistent.After DTW is handled, we take Fourier descriptor that curve is described in turn, Fu that will be obtained In leaf three layers of time pyramid representation are described, and using a quarter of each partial-length as its low frequency coefficient, The Feature Descriptor entirely acted, which, which is advantageous in that, can overcome the unfavorable factors such as noise, time deviation, enhancing Robustness.In terms of action recognition, motion characteristic is fused to convolutional neural networks after completing Lie group feature extraction by such as Fig. 4 In be trained, learn, classify.Using two-dimensional convolution core in convolution process, convolution exports later are as follows:
Wherein, R is activation primitive ReLU, xiFor input feature vector figure, yiTo export characteristic pattern, wi,jJth interlayer is arrived for i-th layer Weight, bjTo set parameter；The structure of entire convolutional neural networks has been divided into 6 layers, and wherein first layer, third layer are convolutional layer C1,C3；The second layer, the 4th layer be maximum pond layer S2, S4；Layer 5, layer 6 are full articulamentum FC5, FC6.Each layer of tool The parameter of body is shown in Fig. 4.
Finally, it is stated that preferred embodiment above is only used to illustrate the technical scheme of the present invention and not to limit it, although logical It crosses above preferred embodiment the present invention is described in detail, however, those skilled in the art should understand that, can be Various changes are made to it in form and in details, without departing from claims of the present invention limited range.

Claims (5)

1. a kind of human motion recognition method based on Lie group feature and convolutional neural networks, which is characterized in that this method is specific The following steps are included:
S1: data acquisition extracts skeleton information using Microsoft somatosensory device Kinect, obtains the motion information of experimenter；
S2: extracting Lie group feature, takes a kind of relative dimensional geometry simulated between each limbs of human body using rigid limbs transformation Human action is modeled as a series of curves in Lie group, and then combines Lie group and Lie algebra by the Lie group bone representation method of relationship Between corresponding relationship, using logarithmic mapping by the curve mapping based on Lie group be the curve based on Lie algebra space；The rigidity Limbs variation includes the rotation and translation of three-dimensional space；
S3: tagsort merges Lie group feature and convolutional neural networks, using Lie group feature training convolutional neural networks, allows volume Product neural network learns Lie group feature, is classified, to realize that human action identifies.
2. the human motion recognition method according to claim 1 based on Lie group feature and convolutional neural networks, feature It is, in the step S1, for the skeleton information of acquisition, is normalized, guarantees the consistency of skeleton size.
3. the human motion recognition method according to claim 2 based on Lie group feature and convolutional neural networks, feature It is, the step S2 is specifically included:
Human skeleton is indicated with S=(V, E), wherein V={ v1,…,vNBe artis set, E={ e1,…,eMIndicate rigid Property limbs set, wherein N is artis quantity, and M is rigid limbs quantity；Define en1∈R3And en2∈R3Respectively indicate limbs enBeginning and end；Given a pair of joint limbs emAnd en, the static state of human body, which acts, can use emAnd enBetween relative geometrical relation It is described, this description method is summarized are as follows: wherein a limbs will be rotated, be moved to and another limb in local coordinate system In the same direction same position of body；Wherein complete rigid limbs conversion process an are as follows: limbs are first around axisWith certain angle Degree θ rotated, rotate to another limbs it is equidirectional after stop rotating, then translate againDistance be allowed to and another limbs It is overlapped.
4. the human motion recognition method according to claim 3 based on Lie group feature and convolutional neural networks, feature It is, in the step S2, the detailed process of the rigidity limbs transformation are as follows:
For rigid limbs emAnd en, by enBy rotating, translating, make itself and emIt is overlapped, obtains emSame enBetween one three-dimensional become Change relationship are as follows:
Wherein,It indicates with limbs emFor starting point, with limbs enFor the rotary shaft vector of terminal；θm,nIndicate limbs emAround axisIt rotates to and limbs enEquidirectional angle；For postrotational limbs emMove to limbs enDistance vector；
Similarly, by emBy rotating, translating, make itself and enIt is overlapped, obtains emSame enBetween another three-dimension varying relationship are as follows:
Wherein,It indicates with limbs emFor starting point, with limbs enFor the rotary shaft vector of terminal；θm,nIndicate limbs emAround axisIt rotates to and limbs enEquidirectional angle；For postrotational limbs emMove to limbs enDistance vector；
Gather the opposite 3D geometrical relationship between all limbs pair, in moment t, a human body skeleton representation is following form:
S (t)=(T1,2(t),T2,1(t),...,TM-1,M(t),TM,M-1(t)) (3)
Wherein, M is rigid limbs quantity, and M (M-1) is the total degree of all rigid limbs transformation, TM,M-1(t) limbs e is indicatedM-1 With limbs eMBetween three-dimension varying relationship；Bone representation more than, the bone sequence for describing human action are expressed as The curve of following form:
{S(t),t∈[0,T']} (4)
Wherein, T ' is total time；
Remember Ri,j(t) it is 3D spin matrix, is expressed asSo (3) can be deformed into:
It will indicate that the curve mapping in Lie group space can be obtained to Lie algebra space:
Wherein, vec (g) indicates vector space.
5. the human motion recognition method according to claim 4 based on Lie group feature and convolutional neural networks, feature It is, is specifically included in the step S3:
S31: the curve obtained in step S2 dynamic time warping (Dynamic Time Warping, DTW) method is carried out Regular processing becomes problem with fix-rate；
S32: after DTW is handled, using Fourier descriptor (Fourier temporal pyramid, FTP) to curve It is described, by one three layers of time pyramid representation of obtained Fourier descriptor, and by four points of each partial-length One of be used as its low frequency coefficient, the Feature Descriptor entirely acted；
S33: after completing Lie group feature extraction, motion characteristic is fused in convolutional neural networks and is trained, learns, dividing Class；Using two-dimensional convolution core in convolution process, convolution exports later are as follows:
Wherein, R is activation primitive ReLU, xiFor input feature vector figure, yiTo export characteristic pattern, wi,jThe power of jth interlayer is arrived for i-th layer Value, bjFor offset parameter.
CN201811446456.5A 2018-11-29 2018-11-29 Human body action recognition method based on lie group features and convolutional neural network Active CN109614899B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811446456.5A CN109614899B (en) 2018-11-29 2018-11-29 Human body action recognition method based on lie group features and convolutional neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811446456.5A CN109614899B (en) 2018-11-29 2018-11-29 Human body action recognition method based on lie group features and convolutional neural network

Publications (2)

Publication Number Publication Date
CN109614899A true CN109614899A (en) 2019-04-12
CN109614899B CN109614899B (en) 2022-07-01

Family

ID=66004965

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811446456.5A Active CN109614899B (en) 2018-11-29 2018-11-29 Human body action recognition method based on lie group features and convolutional neural network

Country Status (1)

Country Link
CN (1) CN109614899B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110188688A (en) * 2019-05-30 2019-08-30 网易（杭州）网络有限公司 Postural assessment method and device
CN110197195A (en) * 2019-04-15 2019-09-03 深圳大学 A kind of novel deep layer network system and method towards Activity recognition
CN110929631A (en) * 2019-11-19 2020-03-27 武汉大学 Scene classification method based on Lie-AdaBoost remote sensing image
CN111126485A (en) * 2019-12-24 2020-05-08 武汉大学 Lie-KFDA scene classification method and system based on Lie group machine learning kernel function
CN111709323A (en) * 2020-05-29 2020-09-25 重庆大学 Gesture recognition method based on lie group and long-and-short term memory network

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010129599A1 (en) * 2009-05-04 2010-11-11 Oblong Industries, Inc. Gesture-based control systems including the representation, manipulation, and exchange of data
CN106445138A (en) * 2016-09-21 2017-02-22 中国农业大学 Human body posture feature extracting method based on 3D joint point coordinates
EP3166033A1 (en) * 2015-11-05 2017-05-10 Samsung Electronics Co., Ltd. Walking assistance apparatus and method of controlling same
CN107169415A (en) * 2017-04-13 2017-09-15 西安电子科技大学 Human motion recognition method based on convolutional neural networks feature coding
CN107229920A (en) * 2017-06-08 2017-10-03 重庆大学 Based on integrating, depth typical time period is regular and Activity recognition method of related amendment
CN107392131A (en) * 2017-07-14 2017-11-24 天津大学 A kind of action identification method based on skeleton nodal distance
CN107506333A (en) * 2017-08-11 2017-12-22 深圳市唯特视科技有限公司 A kind of visual token algorithm based on ego-motion estimation

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010129599A1 (en) * 2009-05-04 2010-11-11 Oblong Industries, Inc. Gesture-based control systems including the representation, manipulation, and exchange of data
EP3166033A1 (en) * 2015-11-05 2017-05-10 Samsung Electronics Co., Ltd. Walking assistance apparatus and method of controlling same
CN106445138A (en) * 2016-09-21 2017-02-22 中国农业大学 Human body posture feature extracting method based on 3D joint point coordinates
CN107169415A (en) * 2017-04-13 2017-09-15 西安电子科技大学 Human motion recognition method based on convolutional neural networks feature coding
CN107229920A (en) * 2017-06-08 2017-10-03 重庆大学 Based on integrating, depth typical time period is regular and Activity recognition method of related amendment
CN107392131A (en) * 2017-07-14 2017-11-24 天津大学 A kind of action identification method based on skeleton nodal distance
CN107506333A (en) * 2017-08-11 2017-12-22 深圳市唯特视科技有限公司 A kind of visual token algorithm based on ego-motion estimation

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
DAN XU等: "Human action recognition based on Kinect and PSO-SVM by representing 3D skeletons as points in lie group", 《2016 INTERNATIONAL CONFERENCE ON AUDIO, LANGUAGE AND IMAGE PROCESSING (ICALIP)》 *
RAVITEJA VEMULAPALLI等: "Rolling Rotations for Recognizing Human Actions from 3D Skeletal Data", 《2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)》 *
杨梦铎: "视觉印象深度学习算法研究", 《中国博士学位论文全文数据库信息科技辑》 *
许欢: "李群机器学习模型及应用研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110197195A (en) * 2019-04-15 2019-09-03 深圳大学 A kind of novel deep layer network system and method towards Activity recognition
WO2020211243A1 (en) * 2019-04-15 2020-10-22 深圳大学 Behavior identification method and apparatus based on deep network technology, and storage medium
CN110197195B (en) * 2019-04-15 2022-12-23 深圳大学 Novel deep network system and method for behavior recognition
CN110188688A (en) * 2019-05-30 2019-08-30 网易（杭州）网络有限公司 Postural assessment method and device
CN110188688B (en) * 2019-05-30 2021-12-14 网易（杭州）网络有限公司 Posture evaluation method and device
CN110929631A (en) * 2019-11-19 2020-03-27 武汉大学 Scene classification method based on Lie-AdaBoost remote sensing image
CN111126485A (en) * 2019-12-24 2020-05-08 武汉大学 Lie-KFDA scene classification method and system based on Lie group machine learning kernel function
CN111709323A (en) * 2020-05-29 2020-09-25 重庆大学 Gesture recognition method based on lie group and long-and-short term memory network
CN111709323B (en) * 2020-05-29 2024-02-02 重庆大学 Gesture recognition method based on Liqun and long-short-term memory network

Also Published As

Publication number Publication date
CN109614899B (en) 2022-07-01

Similar Documents

Publication Publication Date Title
CN109614899A (en) A kind of human motion recognition method based on Lie group feature and convolutional neural networks
CN102999942B (en) Three-dimensional face reconstruction method
CN108830150B (en) One kind being based on 3 D human body Attitude estimation method and device
CN104376594B (en) Three-dimensional face modeling method and device
CN107679522B (en) Multi-stream LSTM-based action identification method
CN104933417B (en) A kind of Activity recognition method based on sparse space-time characteristic
CN101751689B (en) Three-dimensional facial reconstruction method
CN104008564B (en) A kind of human face expression cloning process
WO2017133009A1 (en) Method for positioning human joint using depth image of convolutional neural network
CN110287880A (en) A kind of attitude robust face identification method based on deep learning
Qu et al. A fast face recognition system based on deep learning
CN109214282A (en) A kind of three-dimension gesture critical point detection method and system neural network based
CN107423730A (en) A kind of body gait behavior active detecting identifying system and method folded based on semanteme
CN107871106A (en) Face detection method and device
CN103473801A (en) Facial expression editing method based on single camera and motion capturing data
CN110222580A (en) A kind of manpower 3 d pose estimation method and device based on three-dimensional point cloud
CN108734194A (en) A kind of human joint points recognition methods based on single depth map of Virtual reality
CN114036969B (en) 3D human body action recognition algorithm under multi-view condition
CN107479693A (en) Real-time hand recognition methods based on RGB information, storage medium, electronic equipment
CN107066979A (en) A kind of human motion recognition method based on depth information and various dimensions convolutional neural networks
CN109670401A (en) A kind of action identification method based on skeleton motion figure
CN112906520A (en) Gesture coding-based action recognition method and device
CN111259950A (en) Method for training YOLO neural network based on 3D model
Huang et al. Optimizing features quality: a normalized covariance fusion framework for skeleton action recognition
CN103218611B (en) Based on the human body motion tracking method of distributed collaboration study

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20221027

Address after: 101-2, F1, Building 1, No. 1, Caihefang West Street, Haidian District, Beijing 100080

Patentee after: Beijing Zhifeng Technology Co.,Ltd.

Address before: 400065 Chongqing Nan'an District huangjuezhen pass Chongwen Road No. 2

Patentee before: CHONGQING University OF POSTS AND TELECOMMUNICATIONS