CN110084211A - A kind of action identification method - Google Patents
A kind of action identification method Download PDFInfo
- Publication number
- CN110084211A CN110084211A CN201910362475.8A CN201910362475A CN110084211A CN 110084211 A CN110084211 A CN 110084211A CN 201910362475 A CN201910362475 A CN 201910362475A CN 110084211 A CN110084211 A CN 110084211A
- Authority
- CN
- China
- Prior art keywords
- cluster
- subgroup
- posture
- action
- classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
Abstract
The present invention proposes a kind of action identification method, the three-dimensional skeletal joint point information including obtaining target；It designs parallel link neural network and feature is extracted to every frame skeletal joint point three-dimensional coordinate of action sequence in training set, obtain the feature vector of the frame；By feature vector clusters all in training set at K cluster；Each cluster is calculated to the support of each action classification；Posture subgroup is defined, posture subgroup is extracted from training set, forms posture subgroup set；Study obtains Z hierarchical classification device；The feature vector of the every frame of test action sequence is obtained with parallel link neural network and is allocated to it apart from nearest cluster；Z hierarchical classification device is calculated separately to the classification results of test sample；Classification of the most classification of frequency of occurrence as test action sequence in Z hierarchical classification device classification results of selection.Present invention decreases influence of the difference to action recognition effect in the class of movement, are not influenced by movement generation rate, are capable of handling the action sequence of any time length.
Description
Technical field
The present invention relates to a kind of action identification methods, belong to human action identification technology field.
Background technique
Human action identification is a research hotspot of computer vision field, is widely used in human-computer interaction, video prison
The fields such as control, virtual reality.With the development of depth camera, people can easily obtain the three-dimensional position of skeleton artis
It sets, wherein containing a large amount of motion information.Therefore, the action recognition based on skeletal joint point has caused more and more researchs
The concern of person.Using skeletal joint point data carry out action recognition research, not vulnerable to illumination change, camera angles change etc. because
The influence of element.But due to the influence for the other factors such as blocking of difference in the class of movement, skeletal joint point, to human action
Accurately identifying is still a very challenging task.
Most of existing research work are constructed using hidden Markov model, condition random field or time pyramid
The Time Dynamic Model of skeletal joint point sequence, these research work are extracted the various spies of skeleton artis space structure
Sign, such as pairs of skeletal joint point relative position, skeletal joint point angle, skeletal joint point motion amplitude.These spies extracted
The validity of sign is easy dependent on original bone body joint point coordinate and the accuracy of the information of entire action sequence by individual
The influence of inexact data, and in the case where time scale changes greatly, skeletal joint point space structure is complex, it utilizes
These features are difficult to effective identification maneuver.It is necessary to design a kind of algorithm, the dependence to initial joint point coordinate accuracy is reduced,
And action recognition is carried out based on the key frame in action sequence.
Summary of the invention
The present invention is to solve the problems of the prior art and propose, technical solution is as follows,
A kind of action identification method, includes the following steps:
Step 1: obtaining the three-dimensional skeletal joint point information of target using depth transducer, obtains each bone of human body and close
Obtained action sequence is divided into training set and test set by the three-dimensional coordinate of node；
Step 2: design parallel link neural network model, to each frame skeletal joint of the action sequence in training set
The three-dimensional coordinate of point extracts feature, obtains the feature vector of the frame；
Step 3: by the feature vector clusters of all frames in training set at K cluster；
Step 4: the weight for calculating each cluster and each cluster are to the support of each action classification；
Step 5: defining " posture subgroup ", and posture subgroup is extracted from training set, form posture subgroup set；
Step 6: defining the classifier of the corresponding c class movement of posture subgroup, c ∈ [1, C], C indicate the movement in training set
Classification sum；
Step 7: study obtains Z hierarchical classification device；
Step 8: using parallel link neural network model obtain each frame of test action sequence feature vector and by its
It is allocated to it apart from nearest cluster；
Step 9: calculating separately Z hierarchical classification device to the classification results of test action sequence；
Step 10: the most classification of frequency of occurrence is as test action sequence in Z hierarchical classification device classification results of selection
Classification.
Preferably, the parallel link neural network model in the step 2 is hidden including the first hidden layer, the second hidden layer, third
Layer and the 4th hidden layer, the output of first hidden layerEnter the second hidden layer, the output of the second hidden layer by relu active moduleEnter third hidden layer, the output of the third hidden layer by tanh active moduleEnter the 4th by relu active module
Hidden layer, the output of the 4th hidden layerWith the output of the first hidden layerThe output q of similar computing module is inputted after being added
Tanh active module, carries out Nonlinear Mapping, and the output layer of parallel link neural network model exports the feature vector of the frame；
The input of the parallel link neural network model is by R joint of each frame of the action sequence in training set
One-dimensional vector x=(the x that the three-dimensional coordinate of point is combined into1,x2,...,x3R)T, export as y=(y1,y2,...,y3R)T, first is hidden
Layer, the second hidden layer, third hidden layer, the number of neuron in the 4th hidden layer are respectively N, M, M, N.The calculation of hidden layer are as follows:
Wherein,For the input of hidden layer l, WlFor the weight matrix of hidden layer l, blFor the bias vector of hidden layer l,It is hidden
The output of layer l, l ∈ { 1,2,3,4 }. b1、b4、b2、b3、
Relu, tanh are activation primitive, and preferably relu and tanh, also can be selected other activation primitives herein, all at this
Apply within protection scope.The input of relu active module isOutput isThe element of each dimension of input vector obtains corresponding output in the following manner:
Wherein, d1∈[1,D1]。
The input of tanh active module isOutput isInput
The element of each dimension of vector obtains corresponding output in the following manner:
Wherein, d2∈[1,D2]。
By the output of the first hidden layerThe output of 4th hidden layerAnd the output q of similarity computing module is added, it is defeated
Enter tanh active module, carries out Nonlinear Mapping.The input of similarity computing module is the input of third hidden layerSimilarity measures matrix UT=[u1,u2,…,uN]T, wherein u1、u2、…、uNIt is column vector, un
=[u1n,u2n,...,uMn]T, umn∈ [0,1], m ∈ [1, M], n ∈ [1, N], umnInitial value be set as section [0,1] at random
In any one number.OutputX is acted on obtaining by similarity computing module by two hidden layers, activation primitives
Feature vector and u1、u2、…、uNSimilarity measures are carried out, the dimension of feature vector is increased into N from M.Due to unEach dimension
For the element value of degree between 0 to 1, the output of tanh activation primitive selects the defeated of tanh active module also between 0 to 1
Out as the input of similarity computing module.The calculation of output layer are as follows:
Y=tanh (WOo+bO),
Wherein,For the input of output layer,For weight matrix,For bias vector.
The loss function of parallel link neural network is loss=| | x-y | |2, define action sequence in the frame feature to
Measure the input that f is third hidden layer
Preferably, the step 3 is by the feature vector clusters of all frames in training set at K cluster, and specific steps are such as
Under:
A, a vector is randomly selected from all feature vectors in training set as first cluster centre；
B, calculate the shortest distance of each feature vector and current existing cluster centre, i.e., with a nearest cluster centre
Euclidean distance, and be ranked up from big to small, randomly select one in feature vector corresponding to K distances before coming
As next cluster centre；
C, step b is repeated until selecting K feature vector as K cluster centre；
D, each feature vector in training set is calculated to the Euclidean distance of K cluster centre, and each vector is divided to
Therewith in the corresponding cluster of nearest cluster centre；
E, the center μ of each cluster is recalculatedk, new center is the mean value of all feature vectors in the cluster, calculation formula
Are as follows:
Wherein, nkIndicate the number of feature vector in k-th of cluster, fiIndicate the feature vector in the cluster, i ∈ [1, nk], k
∈[1,K]；
F, defined feature vector f and k-th of cluster distance χkFor this feature vector and the cluster center Euclidean distance and
The sum of the Euclidean distance of 3 feature vectors farthest with its distance, is formulated in this feature vector and the cluster are as follows:
Wherein,For 3 farthest feature vectors of distance f in k-th of cluster；
G, each feature vector is calculated at a distance from K cluster, and is divided to it apart from nearest cluster；
H, the center of each cluster is recalculated；
I, judge whether the center of each cluster changes, if the center of each cluster has not been changed, cluster completion；Otherwise,
G, h are repeated in until the center of all clusters no longer changes.
Preferably, in the step 4 weight of each cluster calculation formula are as follows:
Wherein,nk,cFor the number for the feature vector that c class in k-th of cluster acts, nkFor in k-th of cluster
The number of the feature vector of everything classification, wkFor the weight of k-th of cluster, k ∈ [1, K].
Further, each cluster is as follows to the calculation formula of the support of each action classification in the step 4:
sk,c=wk*rk,c,
Wherein, sk,cThe support that c class is acted for k-th of cluster.
Further, " posture subgroup " is defined in the step 5 refer to will be by NαThe set of a cluster center composition is defined as
One size is NαPosture subgroup Pα, the α posture subgroup be formulated are as follows:
WhereinFor from set { μk| k ∈ [1, K] } in choose NαA cluster center；
Training set one shares J action sequence, for each action sequence V thereinj(1≤j≤J), calculates the sequence
In cluster belonging to each frame, and the center of these clusters is formed into set Ej, J cluster centralization can be obtained altogether；
For each cluster centralization Ej, EjEach nonvoid subset be a posture subgroup, take out all elements number
For 2 or 3 posture subgroup, these posture subgroups are constituted into action sequence VjPosture subgroup set Gj, will be from J cluster center
All posture subgroups taken out in set, are combined into posture subgroup set
Further, posture subgroup is defined in the step 6Point of corresponding c class movement
Class device isIts calculation method is as follows:
1 is set by the action sequence label for belonging to the movement of c class in current training set, the label of remaining action sequence
It is set as 0；The number for remembering action sequence in current training set isFor the movement sequence in current training set
ColumnIf posture subgroup PαIt is contained in action sequenceCluster centralizationI.e.And Pα
In cluster center belonging to the sum of cluster support that c class is acted be greater than threshold θ (Pα), thenOtherwiseIt is formulated are as follows:
Wherein,It indicatesThe support that cluster belonging to respectively acts c class；
θ is chosen, so that classifierMost to everything sequence classification mean error ∈ in current training set
It is small, it is formulated are as follows:
Wherein,Indicate the action sequence in current training setLabel.
Further, to Z hierarchical classification device, specific step is as follows for step 7 middle school acquistion:
A, J action sequence is extracted with putting back at random from training set, as current sub- training set S；
B, L posture subgroup is randomly chosen from posture subgroup set G, and posture subgroup is arranged by the sequence of selection
Sequence forms candidate attitudes subgroup set G'={ P1',P'2,...,P'L, wherein P1',P'2,...,P'LFor from posture subgroup collection
Close the posture subgroup chosen in G；
C, the diversity factor δ of action sequence in current sub- training set S is calculated, calculation formula is as follows:
Wherein, pcIt indicates to belong to ratio shared by the sequence of c class movement in current sub- training set；
D, posture subgroup is chosen from candidate attitudes subgroup set, finds satisfactory classifier, which is made
For the classifier of current sub- training set S, the specific steps are as follows:
First posture subgroup P is chosen from candidate attitudes subgroup set1', since the 1st action classification, calculate it
Corresponding classifierTraining set S is divided into two set: set S1With set S2；S1To meetAction sequence set, S2To meetAction sequence set；
Calculate separately set S1With set S2Diversity factor δ1、δ2, select lesser diversity factor and δ work in the two poor, if poor
Value is greater than threshold value λ, then judges current class deviceMeet division condition, by the posture subgroup from candidate attitudes subgroup
It is rejected in set, and by classifierAs the classifier for dividing set S；
Otherwise, continue to choose next action classification, judge whether its corresponding classifier meets division condition, until looking for
To the classifier for meeting division condition or all action classifications are traversed；
If traversal everything classification does not find the classification for meeting division condition yet for the posture subgroup currently chosen
Device then chooses next posture subgroup in order from candidate attitudes subgroup set, calculates its point for corresponding to different action classifications
Class device, until finding satisfactory classifier.
E, current training set S is divided into set S finding satisfactory classifier1With set S2Afterwards, judgement collects respectively
Close S1With set S2In action sequence whether belong to same category, if being not belonging to same category, current son is combined into the collection
Training set is repeated in step c, d, finds satisfactory classifier, as the corresponding classifier of the set, by the set after
It is continuous to divide, until needing to divide there is no set, a hierarchical classification device F is just obtained at this timez。
F, step a to e is repeated, until obtaining Z hierarchical classification device.
Further, in the step 9, classification method of the Z hierarchical classification device to test action sequence are as follows:
For each hierarchical classification device, first test action sequence is carried out using first sub- training set corresponding classifier
It divides, test action sequence is divided to some set and then continues to draw to it using the corresponding classifier of the set
Point, repeatedly, until can not be subdivided；Training action sequence institute in the set that the test action sequence is divided at this time
The classification of category is the classification belonging to it.
The present invention designs parallel link neural network and carries out feature extraction to the human body attitude of each frame of action sequence, subtracts
Influence of the difference to action recognition effect in the class of small movement；The key frame for extracting action sequence, is moved based on key frame
Work is classified, this influences this method by movement generation rate；It is capable of handling the action sequence of any time length.
Detailed description of the invention
Fig. 1 is a kind of work flow diagram of action identification method of the present invention.
Fig. 2 is parallel link neural network model schematic diagram of the present invention.
Fig. 3 is similarity computing module schematic diagram of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
As shown in Figure 1, a kind of action identification method, the specific steps are as follows:
1, the skeletal joint point information that target is obtained using depth transducer, obtains the three-dimensional of each skeletal joint point of human body
Coordinate.Obtained multiple action sequences are divided into training set and test set, of the action sequence in training set and test set
Number is respectively J=137, T=68.
2, the three-dimensional coordinate of 20 artis of each frame of the action sequence in training set is combined into one-dimensional vector x=
(x1,x2,...,x60)T.Design parallel link neural network model as shown in Figure 2.The input of neural network is x, is exported as y
=(y1,y2,...,y60)T, the first hidden layer, the second hidden layer, third hidden layer, in the 4th hidden layer the number of neuron be respectively 50,
30,30,50.The calculation of hidden layer are as follows:
Wherein,For the input of hidden layer l, WlFor the weight matrix of hidden layer l, blFor the bias vector of hidden layer l,It is hidden
The output of layer l, l ∈ { 1,2,3,4 }. b1、b4、b2、b3、
Relu, tanh are activation primitive, and preferably relu and tanh, also can be selected other activation primitives herein, all at this
Within scope of patent protection.The input of relu active module isOutput isThe element of each dimension of input vector obtains corresponding output in the following manner:
Wherein, d1∈[1,D1]。
The input of tanh active module isOutput isInput
The element of each dimension of vector obtains corresponding output in the following manner:
Wherein, d2∈[1,D2]。
For adding operator, it is by the output of the first hidden layerThe output of 4th hidden layerAnd Similarity measures
The output q of module is added, and inputs tanh active module, carries out Nonlinear Mapping.Similarity computing module as shown in figure 3, it
Input is the input of third hidden layerSimilarity measures matrix UT=[u1,u2,…,u50]T, wherein
u1、u2、…、u50It is column vector, un=[u1n,u2n,...,u30n]T, umn∈ [0,1], m ∈ [1,30], n ∈ [1,50], umn's
Initial value is set as any one number in section [0,1] at random.OutputSimilarity computing module passes through x
The feature vector and u that two hidden layers, activation primitives act on1、u2、…、u50.Similarity measures are carried out, by the dimension of feature vector
Degree increases to 50 from 30.Due to unThe element value of each dimension between 0 to 1, the output of tanh activation primitive also 0 to 1 it
Between, therefore select input of the output of tanh active module as similarity computing module.The calculation of output layer are as follows:
Y=tanh (WOo+bO),
Wherein,For the input of output layer,For weight matrix,For bias vector.
The loss function of parallel link neural network is loss=| | x-y | |2, define action sequence in the frame feature to
Measure the input that f is third hidden layer
3, each frame of each action sequence in training set has corresponding feature vector.It will be all in training set
The feature vector clusters of frame are at K cluster, and K=400, steps are as follows:
Step 1: a vector is randomly selected from all feature vectors in training set as first cluster centre.
Step 2: calculate each feature vector and current existing center the shortest distance (i.e. with a nearest center
Euclidean distance), and be ranked up from big to small, one is randomly selected from coming in feature vector corresponding to preceding 400 distances
It is a to be used as next cluster centre.
Step 3: Step 2 is repeated until selecting 400 feature vectors as 400 cluster centres.
Step 4: calculate training set in each feature vector to 400 cluster centres Euclidean distance, by each vector
It is divided in the corresponding cluster of cluster centre nearest therewith.
Step 5: the center μ of each cluster is recalculatedk, new center is the mean value of all feature vectors in the cluster, is calculated
Formula are as follows:
Wherein, nkIndicate the number of feature vector in k-th of cluster, fiIndicate the feature vector in the cluster, i ∈ [1, nk], k
∈[1,400]。
Step 6: defined feature vector f and k-th of cluster distance χkFor the Euclidean distance of this feature vector and the cluster center
And the sum of the Euclidean distance in this feature vector and the cluster with its 3 feature vectors of distance farthest, it is formulated are as follows:
Wherein,For 3 farthest feature vectors of distance f in k-th of cluster.
Step 7: each feature vector is calculated at a distance from 400 clusters, and is divided to it apart from nearest cluster.
Step 8: the center of each cluster is recalculated.
Step 9: judging whether the center of each cluster changes, if the center of each cluster has not been changed, clusters completion.
Otherwise, Step 7,8 is repeated in until the center of all clusters no longer changes.
It as above, can be by the corresponding feature vector clusters of all frames in training set at 400 clusters.
4, the weight w of each cluster is calculatedk, calculation formula are as follows:
Wherein,nk,cFor the number for the feature vector that c class in k-th of cluster acts, c ∈ [1, C], C are indicated
Action classification sum in training set；Set C=8.
5, the support s that k-th of cluster acts c class is definedk,cFor the weight w of the clusterkIt is acted with c class in the cluster
Feature vector proportion rk,cProduct, calculation formula is as follows:
sk,c=wk*rk,c。
As above, each cluster can be calculated to the support of different action classifications.
6, " posture subgroup " is defined: will be by NαIt is N that the set of a cluster center composition, which is defined as a size,αPosture subgroup
Pα, it is formulated are as follows:
Wherein,For from set { μk| k ∈ [1,400] } in choose NαA cluster center.
Training set one shares J=137 action sequence, for each action sequence V thereinj(1≤j≤137) calculate
Cluster belonging to each frame in the sequence, and the center of these clusters is formed into set Ej, 137 cluster centralizations can be obtained altogether.
For each cluster centralization Ej, EjEach nonvoid subset be a posture subgroup, take out all elements number
For 2 or 3 posture subgroup, these posture subgroups are constituted into action sequence VjPosture subgroup set Gj.It will be from 137 clusters
All posture subgroups taken out in heart set, are combined into posture subgroup set
7, posture subgroup is definedCorresponding to the classifier that c class acts isIts
Calculation method is as follows:
1 is set by the action sequence label for belonging to the movement of c class in current training set, the label of remaining action sequence
It is set as 0.The number for remembering action sequence in current training set isFor the movement in current training set
SequenceIf posture subgroup PαIt is contained in action sequenceCluster centralizationI.e.And
PαIn cluster center belonging to the sum of cluster support that c class is acted be greater than threshold θ (Pα), thenOtherwiseIt is formulated are as follows:
Wherein,It indicatesThe support that cluster belonging to respectively acts c class.
θ is chosen, so that classifierMost to everything sequence classification mean error ∈ in current training set
It is small, it is formulated are as follows:
Wherein,Indicate the action sequence in current training setLabel.
8,137 action sequences are extracted with putting back at random from training set, as current sub- training set S, it is carried out
Level divides, and obtains classifier Fz, steps are as follows:
Step 1: 40 posture subgroups are randomly chosen from posture subgroup set G, by the sequence of selection to posture subgroup
It is ranked up, forms candidate attitudes subgroup set G'={ P1',P'2,...,P'40, wherein P1',P'2,...,P'40For from posture
The posture subgroup chosen in subgroup set.
Step 2: calculating the diversity factor δ of action sequence in current sub- training set S, and calculation formula is as follows:
Wherein, pcIt indicates to belong to ratio shared by the sequence of c class movement in current sub- training set.
Step 3: posture subgroup is chosen from candidate attitudes subgroup set, satisfactory classifier is found, by the classification
Classifier of the device as current sub- training set S, is 2 set by S points, steps are as follows:
First posture subgroup P is chosen from candidate attitudes subgroup set1', since the 1st action classification, calculate it
Corresponding classifierTraining set S is divided into two set: set S1With set S2。S1To meetAction sequence set, S2To meetAction sequence set.
Calculate separately set S1With set S2Diversity factor δ1、δ2, select lesser diversity factor and δ work in the two poor, if poor
Value is greater than 0.1, then judges current class deviceMeet division condition, by the posture subgroup from candidate attitudes subgroup collection
It is rejected in conjunction, and by classifierAs the classifier for dividing set S.
Otherwise, continue to choose next action classification, judge whether its corresponding classifier meets division condition, until looking for
To the classifier for meeting division condition or all action classifications are traversed.
If traversal everything classification does not find the classification for meeting division condition yet for the posture subgroup currently chosen
Device then chooses next posture subgroup in order from candidate attitudes subgroup set, calculates its point for corresponding to different action classifications
Class device, until finding satisfactory classifier.
Step 4: current training set S is divided into set S finding satisfactory classifier1With set S2Afterwards, respectively
Judge set S1With set S2In action sequence whether belong to same category, if being not belonging to same category, be combined into the collection
Current sub- training set, is repeated in Step 2,3, which is continued to divide, until needing to divide there is no set.At this time just
Obtain a hierarchical classification device Fz。
9, step 8 is repeated, until obtaining 20 hierarchical classification devices.
10, that the three-dimensional coordinate of 20 artis of each frame of everything sequence in test set is connected into one-dimensional vector is defeated
Enter to obtain to trained parallel link neural network model the feature of the human body attitude of each frame of all test samples to
Amount.
11, for some test sample, the feature vector of each frame of its action sequence is calculated at a distance from each cluster, and will
It is allocated to it apart from nearest cluster.
12, classified using 20 obtained hierarchical classification devices to test action sequence:
For each hierarchical classification device, first test action sequence is carried out using first sub- training set corresponding classifier
It divides, test action sequence is divided to some set and then continues to draw to it using the corresponding classifier of the set
Point, repeatedly, until can not be subdivided.Training action sequence institute in the set that the test action sequence is divided at this time
The classification of category is the classification belonging to it.
13, the class that the classification that frequency of occurrence is most in 20 hierarchical classification device classification results is test action sequence is chosen
Not.
Although the present invention is described in detail referring to the foregoing embodiments, for those skilled in the art,
It is still possible to modify the technical solutions described in the foregoing embodiments, or part of technical characteristic is carried out etc.
With replacement, all within the spirits and principles of the present invention, any modification, equivalent replacement, improvement and so on should be included in this
Within the protection scope of invention.
Claims (9)
1. a kind of action identification method, includes the following steps:
Step 1: obtaining the three-dimensional skeletal joint point information of target using depth transducer, each skeletal joint point of human body is obtained
Three-dimensional coordinate, obtained action sequence is divided into training set and test set；
Step 2: design parallel link neural network model, to each frame skeletal joint point of the action sequence in training set
Three-dimensional coordinate extracts feature, obtains the feature vector of each frame；
Step 3: by the feature vector clusters of all frames in training set at K cluster；
Step 4: the weight for calculating each cluster and each cluster are to the support of each action classification；
Step 5: defining " posture subgroup ", and posture subgroup is extracted from training set, form posture subgroup set；
Step 6: defining the classifier of the corresponding c class movement of posture subgroup, c ∈ [1, C], C indicate the action classification in training set
Sum；
Step 7: study obtains Z hierarchical classification device；
Step 8: obtaining the feature vector of each frame of test action sequence using parallel link neural network model and being divided
It is given apart from nearest cluster；
Step 9: calculating separately Z hierarchical classification device to the classification results of test action sequence；
Step 10: class of the most classification of frequency of occurrence as test action sequence in Z hierarchical classification device classification results of selection
Not.
2. a kind of action identification method according to claim 1, it is characterised in that: the parallel link mind in the step 2
It include the first hidden layer, the second hidden layer, third hidden layer and the 4th hidden layer, the output of first hidden layer through network modelPass through
Relu active module enters the second hidden layer, the output of the second hidden layerEnter third hidden layer by tanh active module, described
The output of three hidden layersEnter the 4th hidden layer, the output of the 4th hidden layer by relu active moduleWith the first hidden layer
OutputThe output q of similar computing module inputs tanh active module after being added, and carries out Nonlinear Mapping, parallel link nerve
The output layer of network model exports the feature vector of the frame；
The input of the parallel link neural network model is the R artis by each frame of the action sequence in training set
One-dimensional vector x=(the x that three-dimensional coordinate is combined into1,x2,...,x3R)T, export as y=(y1,y2,...,y3R)T, the first hidden layer,
Second hidden layer, third hidden layer, the number of neuron in the 4th hidden layer are respectively N, M, M, N, the calculation of each hidden layer output
Are as follows:
WhereinFor the input of hidden layer l, WlFor the weight matrix of hidden layer l, blFor the bias vector of hidden layer l,For the defeated of hidden layer l
Out, { 1,2,3,4 } l ∈, b1、b4、b2、b3、
The input of relu active module isOutput isInput vector
The element of each dimension obtain corresponding output in the following manner:
Wherein, [1, D] d ∈；
The input of tanh active module isOutput isInput vector
The element of each dimension obtains corresponding output in the following manner:
Wherein, [1, D'] d' ∈；
The output of the similarity computing moduleThe input of similarity computing module is the input of third hidden layerSimilarity measures matrix UT=[u1,u2,…,uN]T, wherein u1、u2、…、uNIt is column vector, un
=[u1n,u2n,...,uMn]T, umn∈ [0,1], m ∈ [1, M], n ∈ [1, N], umnInitial value be set as section [0,1] at random
In any one number；The calculation of the output layer of parallel link neural network are as follows:
Y=tanh (WOo+bO),
Wherein,For the input of output layer,For weight matrix,For bias vector；
The loss function of parallel link neural network is loss=| | x-y | |2, defining the feature vector f of the frame in action sequence is
The input of third hidden layer
3. a kind of action identification method according to claim 1, it is characterised in that: the step 3 is by the institute in training set
There are the feature vector clusters of frame at K cluster, the specific steps are as follows:
A, a vector is randomly selected from all feature vectors in training set as first cluster centre；
B, the shortest distance of each feature vector and current existing cluster centre, the i.e. Europe with a nearest cluster centre are calculated
Formula distance, and be ranked up from big to small, a conduct is randomly selected in feature vector corresponding to K distances before coming
Next cluster centre；
C, step b is repeated until selecting K feature vector as K cluster centre；
D, each feature vector in training set is calculated to the Euclidean distance of K cluster centre, and each vector is divided to therewith
In the corresponding cluster of nearest cluster centre；
E, the center μ of each cluster is recalculatedk, new center is the mean value of all feature vectors in the cluster, calculation formula are as follows:
Wherein, nkIndicate the number of feature vector in k-th of cluster, fiIndicate the feature vector in the cluster, i ∈ [1, nk], k ∈ [1,
K]；
F, defined feature vector f and k-th of cluster distance χkFor the Euclidean distance and this feature of this feature vector and the cluster center
The sum of the Euclidean distance of 3 feature vectors farthest with its distance, is formulated in vector and the cluster are as follows:
Wherein,For 3 farthest feature vectors of distance f in k-th of cluster；
G, each feature vector is calculated at a distance from K cluster, and is divided to it apart from nearest cluster；
H, the center of each cluster is recalculated；
I, judge whether the center of each cluster changes, if the center of each cluster has not been changed, cluster completion；Otherwise, successively
G, h are repeated until the center of all clusters no longer changes.
4. a kind of action identification method according to claim 1, it is characterised in that: the weight of each cluster in the step 4
Calculation formula are as follows:
Wherein,nk,cFor the number for the feature vector that c class in k-th of cluster acts, nkIt is all dynamic in k-th of cluster
Make the number of the feature vector of classification, wkFor the weight of k-th of cluster, k ∈ [1, K], c ∈ [1, C], C indicate dynamic in training set
Make classification sum.
5. according to a kind of action identification method as claimed in claim 4, it is characterised in that: each cluster is to each dynamic in the step 4
The calculation formula for making the support of classification is as follows:
sk,c=wk*rk,c,
Wherein, sk,cThe support that c class is acted for k-th of cluster.
6. according to a kind of action identification method as claimed in claim 3, it is characterised in that: define " posture subgroup " in the step 5
Referring to will be by NαIt is N that the set of a cluster center composition, which is defined as a size,αPosture subgroup Pα, the α posture subgroup formula
It indicates are as follows:
Wherein,For from set { μk| k ∈ [1, K] } in choose NαA cluster center；
Training set one shares J action sequence, for each action sequence V thereinj(1≤j≤J) is calculated each in the sequence
Cluster belonging to frame, and the center of these clusters is formed into set Ej, J cluster centralization can be obtained altogether；
For each cluster centralization Ej, EjEach nonvoid subset be a posture subgroup, take out all elements number be 2
Or 3 posture subgroup, by these posture subgroups constitute action sequence VjPosture subgroup set Gj, will be from J cluster centralization
All posture subgroups of middle taking-up, are combined into posture subgroup set
7. according to a kind of action identification method as claimed in claim 6, it is characterised in that: define posture subgroup in the step 6Corresponding to the classifier that c class acts isIts calculation method is as follows:
1 is set by the action sequence label for belonging to the movement of c class in current training set, the label setting of remaining action sequence
It is 0；The number for remembering action sequence in current training set isFor the action sequence in current training setIf posture subgroup PαIt is contained in action sequenceCluster centralizationI.e.And PαIn
Cluster center belonging to the sum of cluster support that c class is acted be greater than threshold θ (Pα), thenOtherwiseIt is formulated are as follows:
Wherein,It indicatesThe support that cluster belonging to respectively acts c class；
θ is chosen, so that classifierIt is minimum to everything sequence classification mean error ∈ in current training set, it uses
Formula indicates are as follows:
Wherein, Indicate the action sequence in current training set's
Label.
8. according to a kind of action identification method as claimed in claim 7, it is characterised in that: step 7 middle school acquistion to Z layer
Specific step is as follows for grade classifier:
A, J action sequence is extracted with putting back at random from training set, as current sub- training set S；
B, L posture subgroup is randomly chosen from posture subgroup set G, and posture subgroup is ranked up by the sequence of selection,
Form candidate attitudes subgroup set G'={ P1',P'2,...,P'L, wherein P1',P'2,...,P'LFor from posture subgroup set G
The posture subgroup of middle selection；
C, the diversity factor δ of action sequence in current sub- training set S is calculated, calculation formula is as follows:
Wherein, pcIt indicates to belong to ratio shared by the sequence of c class movement in current sub- training set；
D, posture subgroup is chosen from candidate attitudes subgroup set, finds satisfactory classifier, using the classifier as working as
The classifier of preceding sub- training set S, the specific steps are as follows:
First posture subgroup P is chosen from candidate attitudes subgroup set1', since the 1st action classification, it is corresponding to calculate its
ClassifierTraining set S is divided into two set: set S1With set S2；S1To meetIt is dynamic
Make arrangement set, S2To meetAction sequence set；
Calculate separately set S1With set S2Diversity factor δ1、δ2, select lesser diversity factor and δ work in the two poor, if difference is big
In threshold value λ, then current class device is judgedMeet division condition, by the posture subgroup from candidate attitudes subgroup set
Middle rejecting, and by classifierAs the classifier for dividing set S；
Otherwise, continue to choose next action classification, judge whether its corresponding classifier meets division condition, until finding symbol
It closes the classifier of division condition or has traversed all action classifications；
If traversal everything classification does not find the classifier for meeting division condition yet for the posture subgroup currently chosen,
It then chooses next posture subgroup in order from candidate attitudes subgroup set, calculates its classification for corresponding to different action classifications
Device, until finding satisfactory classifier；
E, current training set S is divided into set S finding satisfactory classifier1With set S2Afterwards, set S is judged respectively1
With set S2In action sequence whether belong to same category, if being not belonging to same category, current son instruction is combined into the collection
Practice collection, is repeated in step c, d, finds satisfactory classifier, as the corresponding classifier of the set, which is continued
It divides, until needing to divide there is no set, just obtains a hierarchical classification device F at this timez；
F, step a to e is repeated, until obtaining Z hierarchical classification device.
9. according to a kind of action identification method according to any one of claims 8, it is characterised in that: in the step 9, Z hierarchical classification device
To the classification method of test action sequence are as follows:
For each hierarchical classification device, first test action sequence is drawn using first sub- training set corresponding classifier
Point, test action sequence is divided to some set and then continues to divide to it using the corresponding classifier of the set,
It repeatedly, until can not be subdivided；Belonging to the training action sequence in set that the test action sequence is divided at this time
Classification be classification belonging to it.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910362475.8A CN110084211B (en) | 2019-04-30 | 2019-04-30 | Action recognition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910362475.8A CN110084211B (en) | 2019-04-30 | 2019-04-30 | Action recognition method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110084211A true CN110084211A (en) | 2019-08-02 |
| CN110084211B CN110084211B (en) | 2020-12-18 |
Family
ID=67418132
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910362475.8A Active CN110084211B (en) | 2019-04-30 | 2019-04-30 | Action recognition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110084211B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112364922A (en) * | 2020-11-13 | 2021-02-12 | 苏州浪潮智能科技有限公司 | Method and system for predicting human skeleton motion in machine room environment |
| CN112712003A (en) * | 2020-12-25 | 2021-04-27 | 华南理工大学 | Joint tag data enhancement method for bone action sequence recognition |
| CN113591797A (en) * | 2021-08-23 | 2021-11-02 | 苏州大学 | Deep video behavior identification method |
| WO2023173553A1 (en) * | 2022-03-15 | 2023-09-21 | 平安科技（深圳）有限公司 | Clustering algorithm-based pedestrian abnormal behavior detection method and apparatus, and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102663449A (en) * | 2012-03-12 | 2012-09-12 | 西安电子科技大学 | Method for tracing human body movement based on maximum geometric flow histogram |
| CN103198492A (en) * | 2013-03-28 | 2013-07-10 | 沈阳航空航天大学 | Human motion capture method |
| CN104318248A (en) * | 2014-10-21 | 2015-01-28 | 北京智谷睿拓技术服务有限公司 | Action recognition method and action recognition device |
| US9679226B1 (en) * | 2012-03-29 | 2017-06-13 | Google Inc. | Hierarchical conditional random field model for labeling and segmenting images |
| US20180174001A1 (en) * | 2016-12-15 | 2018-06-21 | Samsung Electronics Co., Ltd. | Method of training neural network, and recognition method and apparatus using neural network |
| CN108537145A (en) * | 2018-03-21 | 2018-09-14 | 东北电力大学 | Human bodys' response method based on space-time skeleton character and depth belief network |
| CN108681700A (en) * | 2018-05-04 | 2018-10-19 | 苏州大学 | A kind of complex behavior recognition methods |
-
2019
- 2019-04-30 CN CN201910362475.8A patent/CN110084211B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102663449A (en) * | 2012-03-12 | 2012-09-12 | 西安电子科技大学 | Method for tracing human body movement based on maximum geometric flow histogram |
| US9679226B1 (en) * | 2012-03-29 | 2017-06-13 | Google Inc. | Hierarchical conditional random field model for labeling and segmenting images |
| CN103198492A (en) * | 2013-03-28 | 2013-07-10 | 沈阳航空航天大学 | Human motion capture method |
| CN104318248A (en) * | 2014-10-21 | 2015-01-28 | 北京智谷睿拓技术服务有限公司 | Action recognition method and action recognition device |
| US20180174001A1 (en) * | 2016-12-15 | 2018-06-21 | Samsung Electronics Co., Ltd. | Method of training neural network, and recognition method and apparatus using neural network |
| CN108537145A (en) * | 2018-03-21 | 2018-09-14 | 东北电力大学 | Human bodys' response method based on space-time skeleton character and depth belief network |
| CN108681700A (en) * | 2018-05-04 | 2018-10-19 | 苏州大学 | A kind of complex behavior recognition methods |
Non-Patent Citations (2)
| Title |
|---|
| ALESSIA SAGGESE ET AL.: "Learning skeleton representations for human action recognition", 《PATTERN RECOGNITION LETTERS》 * |
| 崔广才等: "基于傅里叶与局部特征结合的", 《长春理工大学学报（自然科学版）》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112364922A (en) * | 2020-11-13 | 2021-02-12 | 苏州浪潮智能科技有限公司 | Method and system for predicting human skeleton motion in machine room environment |
| CN112364922B (en) * | 2020-11-13 | 2023-01-10 | 苏州浪潮智能科技有限公司 | Method and system for predicting human skeleton motion in machine room environment |
| CN112712003A (en) * | 2020-12-25 | 2021-04-27 | 华南理工大学 | Joint tag data enhancement method for bone action sequence recognition |
| CN113591797A (en) * | 2021-08-23 | 2021-11-02 | 苏州大学 | Deep video behavior identification method |
| CN113591797B (en) * | 2021-08-23 | 2023-07-28 | 苏州大学 | Depth video behavior recognition method |
| WO2023173553A1 (en) * | 2022-03-15 | 2023-09-21 | 平安科技（深圳）有限公司 | Clustering algorithm-based pedestrian abnormal behavior detection method and apparatus, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110084211B (en) | 2020-12-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110084211A (en) | A kind of action identification method | |
| CN106682598A (en) | Multi-pose facial feature point detection method based on cascade regression | |
| CN110287880A (en) | A kind of attitude robust face identification method based on deep learning | |
| CN110490177A (en) | A kind of human-face detector training method and device | |
| CN107766850A (en) | Based on the face identification method for combining face character information | |
| CN107229904A (en) | A kind of object detection and recognition method based on deep learning | |
| CN107832672A (en) | A kind of pedestrian's recognition methods again that more loss functions are designed using attitude information | |
| CN107871106A (en) | Face detection method and device | |
| CN110309861A (en) | A kind of multi-modal mankind's activity recognition methods based on generation confrontation network | |
| CN106529448A (en) | Method for performing multi-visual-angle face detection by means of integral channel features | |
| CN109558902A (en) | A kind of fast target detection method | |
| CN106897738A (en) | A kind of pedestrian detection method based on semi-supervised learning | |
| CN111062340B (en) | Abnormal gait behavior recognition method based on virtual gesture sample synthesis | |
| CN106203487A (en) | A kind of image classification method based on Multiple Kernel Learning Multiple Classifier Fusion and device | |
| CN105354593B (en) | A kind of threedimensional model sorting technique based on NMF | |
| CN105740915B (en) | A kind of collaboration dividing method merging perception information | |
| CN109086660A (en) | Training method, equipment and the storage medium of multi-task learning depth network | |
| CN109033953A (en) | Training method, equipment and the storage medium of multi-task learning depth network | |
| CN108537145A (en) | Human bodys' response method based on space-time skeleton character and depth belief network | |
| CN107424161A (en) | A kind of indoor scene image layout method of estimation by thick extremely essence | |
| CN110334584B (en) | Gesture recognition method based on regional full convolution network | |
| CN104598889B (en) | The method and apparatus of Human bodys' response | |
| CN109902585A (en) | A kind of three modality fusion recognition methods of finger based on graph model | |
| CN111311702B (en) | Image generation and identification module and method based on BlockGAN | |
| CN104751463B (en) | A kind of threedimensional model optimal viewing angle choosing method based on sketch outline feature |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |