CN107229920A - Based on integrating, depth typical time period is regular and Activity recognition method of related amendment - Google Patents
Based on integrating, depth typical time period is regular and Activity recognition method of related amendment Download PDFInfo
- Publication number
- CN107229920A CN107229920A CN201710425906.1A CN201710425906A CN107229920A CN 107229920 A CN107229920 A CN 107229920A CN 201710425906 A CN201710425906 A CN 201710425906A CN 107229920 A CN107229920 A CN 107229920A
- Authority
- CN
- China
- Prior art keywords
- mrow
- mtd
- msub
- mtr
- bone
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 74
- 230000000694 effects Effects 0.000 title claims abstract description 54
- 210000000988 bone and bone Anatomy 0.000 claims abstract description 72
- 238000006073 displacement reaction Methods 0.000 claims abstract description 36
- 239000011159 matrix material Substances 0.000 claims abstract description 17
- 238000013507 mapping Methods 0.000 claims abstract description 15
- 238000013519 translation Methods 0.000 claims abstract description 10
- 238000012545 processing Methods 0.000 claims abstract description 7
- 238000012706 support-vector machine Methods 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 22
- 238000012549 training Methods 0.000 claims description 17
- 230000009466 transformation Effects 0.000 claims description 17
- 230000006399 behavior Effects 0.000 claims description 15
- 238000012937 correction Methods 0.000 claims description 15
- 230000033001 locomotion Effects 0.000 claims description 14
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 230000008859 change Effects 0.000 claims description 9
- 239000013598 vector Substances 0.000 claims description 9
- 230000009467 reduction Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000010354 integration Effects 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 5
- 230000004913 activation Effects 0.000 claims description 4
- 238000004458 analytical method Methods 0.000 claims description 4
- 238000000844 transformation Methods 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000010191 image analysis Methods 0.000 claims description 3
- 230000010363 phase shift Effects 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 230000000875 corresponding effect Effects 0.000 description 14
- 101001138022 Homo sapiens La-related protein 1 Proteins 0.000 description 10
- 101000650621 Homo sapiens Septin-1 Proteins 0.000 description 10
- 102100020859 La-related protein 1 Human genes 0.000 description 10
- 230000009471 action Effects 0.000 description 9
- 238000012360 testing method Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000010219 correlation analysis Methods 0.000 description 7
- 238000013135 deep learning Methods 0.000 description 7
- 238000012986 modification Methods 0.000 description 6
- 230000004048 modification Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000004927 fusion Effects 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 210000003423 ankle Anatomy 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 239000003651 drinking water Substances 0.000 description 1
- 235000020188 drinking water Nutrition 0.000 description 1
- 210000000245 forearm Anatomy 0.000 description 1
- 230000005021 gait Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 210000003127 knee Anatomy 0.000 description 1
- 210000002414 leg Anatomy 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 230000001603 reducing effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 210000000707 wrist Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
Abstract
Based on integrating, depth typical time period is regular and Activity recognition method of related amendment the present invention relates to a kind of, what is solved is that recognition accuracy is low, the technical problem that time-consuming, it is rigid body displacement by using by the behavior representation of human body, rigid body displacement is decomposed into translation of rigid body and rigid body rotation, rigid body displacement is represented with homogeneous matrix Lie group SE (3), Lie algebra is SO (3), the skeleton data of collection sets up skeleton model C (t), displacement mapping relations are expressed as homogeneous matrix Lie group SE (3), set up the skeleton model C (t) that method is described based on Lie algebra relative characteristic, and difference processing is carried out to skeleton model C (t)；Alignd using the regular method of depth typical time period is integrated；Using correlative character, alignment feature sample is corrected, the technical scheme classified using SVMs to revised feature samples preferably resolves the problem, in the Activity recognition for 3D bones.
Description
Technical field
The present invention relates to Human bodys' response field, and in particular to depth typical time period is regular and correlation based on integrating for one kind
The Activity recognition method of amendment.
Background technology
Human behavior identification belongs to computer vision and behavior pattern recognition field, and one has been increasingly becoming at nearest several years
Individual heat subject.In terms of man-machine interaction, safety and protection monitoring and content based video retrieval system, Activity recognition has huge
Actual use value, the effect of Activity recognition is just becoming increasingly conspicuous its important function with information-based construction.Meanwhile, Activity recognition
Also there is certain facilitation to the other field of computer vision, such as recognition of face, gait analysis.Past is due to science and technology
Limitation, most of Activity recognition is all based on 2D video and image, and the effect of identification is all undesirable.As depth camera is first-class
The development and popularization of 3D technology, human behavior identification also gradually develop from the feature recognition of plane to the feature recognition of facade, 3D
Activity recognition turns into the main approaches that human behavior is recognized.During motion feature is gathered, different movement velocitys
The data produced can be caused to occur larger difference in time series.Therefore during Activity recognition, alignment of data is
Influence the key factor of the accuracy rate of identification.
Existing 3D Activity recognitions optical flow estimation, 3D skeleton models, 3D skeleton patterns, space-time characteristic.It is existing to be based on 3D
The Activity recognition method of bone uses multiple cameras or other sensors, the articulate position of each frame institute in entirely moving
Confidence breath is showed with the mode of coordinate, and the arrangement of the data of each frame in time series just constitutes the complete of this action
Portion's information.This is a kind of character description method based on point feature.Human body can regard one as and is hinged by what rigid body section was constituted and be
System, tie point is exactly the joint of human body, in computer vision, and substantial amounts of research is local by extracting joint information or detection
Body part recognizes behavior.The method that the body segmentation of the mankind is carried out to Activity recognition into several parts, from single image
It is middle extract human body joint position information, and by human body be divided into head, neck, shoulder, arm, ancon, wrist, hand, trunk, leg,
The parts such as knee, ankle, different actions can be represented with different parts.This method excites computer vision field
The interest of scientific research personnel, the focus in Activity recognition field is turned into using the being hinged property identification human behavior of bone.But, existing skill
There is the problem of accuracy of identification is low, recognition time is long in art.Therefore it provides a kind of high behavior based on 3D bones of accuracy of identification is known
Other method is just necessary.
The content of the invention
The technical problems to be solved by the invention are to examine that accuracy of identification is low, time length technology is asked present in prior art
Topic.A kind of new Activity recognition method based on 3D bones is provided, the Activity recognition method based on 3D bones of being somebody's turn to do has identification essence
The characteristics of spending high.
In order to solve the above technical problems, the technical scheme used is as follows：
It is a kind of that based on integrating, depth typical time period is regular and Activity recognition method of related amendment, including image preprocessing,
Graphical analysis, image understanding, described image analysis include：
(1) it is rigid body displacement by the behavior representation of human body, rigid body displacement is decomposed into translation of rigid body and rigid body rotates, with neat
Submatrix Lie group SE (3) represents rigid body displacement, and reaction homogeneous matrix Lie group is that the Lie algebra of SE (3) full detail is SO (3)；
(2) skeleton data of collection, which is set up in skeleton model C (t), skeleton model C (t), N sections of bone rigid bodies, in t
Carve, one section of bone is defined as to the result of another section of bone displacement, the position relationship between one section of bone and another section of bone is determined
Justice is displacement mapping relations, and in theorem in Euclid space, displacement mapping relations are expressed as into homogeneous matrix Lie group SE (3), then will be neat
The corresponding Lie algebra SO (3) of submatrix Lie group SE (3) are as motion feature, and foundation describes method based on Lie algebra relative characteristic
Skeleton model C (t), and difference processing is carried out to skeleton model C (t), Lie algebra relative characteristic describes method i.e. Lie Algebra
Relative pairs, abbreviation LARP,
Wherein, N is positive integer；
(3) using the depth typical time period regular method alignment skeleton model C (t), the feature samples after being alignd；
(4) alignment feature sample in correlative character, amendment step (3) is utilized, revised alignment feature sample is obtained；
(5) revised feature samples are classified using SVMs, obtains Activity recognition result.
The operation principle of the present invention：In order to strengthen the effect of Activity recognition, we are utilized after Relative modification alignment
Test sample, obtains more preferable test result.Mainly by training, depth typical time period is regular to make the correlation between sample
Property is improved.
In such scheme, for optimization, further, the regular method of depth typical time period is regular to integrate depth typical time period
Method.The main deep layer network structure improved in the regular method of depth typical time period, i.e. DCTW, it is therefore an objective to which appropriate reduction is calculated
The run time of method.During DCTW is applied into LARP, non-linear change is all carried out to every group of data in training sample
Although the accuracy of Activity recognition can be improved by changing, it is due to that calculative gradient is excessive, causes the operation time mistake of algorithm
It is long, therefore other straightforward procedures are found in our trials.The purpose of hands-on is to find one group of general data characteristics, right
Each group of data in sample ask loss function to be not essential, and we are obtained first with the average method of dynamic time warping in trial
To one group of data, the amount of calculation that this is organized needs in the representative counting loss function as whole samples, such deep layer network is obtained
Significant improvement has been arrived, the calculating time can be reduced.
Further, step (4) described correlative character include by skeleton model C (t) with align after feature samples pair
Together, the correlation of each frame of feature samples after calculating skeleton model C (t) and align, is variograph using coefficient correlation x ∈ R
Correction factor is calculated, the weight for the corresponding frame of change that correction factor and the characteristic of each frame are multiplied, correction factor obeys amendment
Function ex。
Further, step (3) the regular method of depth typical time period of integrating includes：
Nonlinear transformation is carried out to skeleton model C (t) using the activation primitive of weight W (t) and deep layer network, converted
Sample C ' (t), using first group of data C ' 1 in conversion sample C ' (t) as normal data, will be become using dynamic time warping method
Vary this C ' (t) alignd respectively with normal data C ' 1, be averaging after obtain integration characteristics C, according to closing feature C and normal data
C ' 1 and：
Calculate covariance, and carry out SVD and decompose to obtain U, V, according to U, V, conversion sample C, (t) and
Calculate and conversion sample C, (t) one-to-one gradient G (t), according to the study of gradient G (t) and deep layer network
Rate updates weight W (t), after convergence, by the training characteristics C of all process deep layer network transformations1f, C2f, C3f...-and normal data
The corresponding training characteristics C of C ' 1fAverage after being alignd with dynamic time warping, result of calculation is alignment feature sample；
Wherein, total number of plies of deep layer network is at least 2.
Further, the deep layer network is BP networks, and it is hidden that BP networks include an input layer, an output layer and two
Layer.
Further, step (2) the difference processing is including the use of interpolation method, and the interpolation method includes：By inserting
Frame number identical feature is worth to, interpolation formula is：
Wherein,Define Q1, Q2, Q3…Qn∈ SE (3) are respectively t1, t2, t3…tn
The instantaneous relative position at moment.
Further, the skeleton model C that method is described based on Lie algebra relative characteristic is set up described in the step (2)
(t) include：Select the e in skeleton modelmBone and enBone is analyzed, emThe end points of bone is em1And em2, enThe end of bone
Point is en1And en2, with end points en1For origin, enThe direction vector of bone is axle, bone emWith bone enResiding face is built for coordinate surface
Vertical coordinate system, emThe end points e of bonem1And em2Relative position be expressed as：
enEnd points relative to emPosition be：
emBone and enThe displacement mapping relations of bone are emBone and enThe relative position relation of bone is：
According to emBone and enThe relative position relation of bone calculates each bone and the phase shift mapping relations of other bones
Ve (B), defining skeleton model has n sections of bone rigid bodies, show that the skeleton model of t is shaped as C (t) and is：
C (t)=(ve (B1,2), ve (B1,3) ..., ve (BN, n-1)),
Ve (B)=(u1, u2, u3, v1, v2, v3)
Wherein, C (t) has 6 × n × (n-1) individual vector, and n and m is the positive integer less than N.
Further, described image pretreatment also includes image noise reduction and image enhaucament.
The information that the logarithm of one matrix Lie group can be constituted in a Lie algebra space, Lie group can be fully converted into Lee's generation
Several forms.The displacement of rigid body can be represented with Lie group, the shift transformation of rigid body can resolve into translation and rotation two parts.It is false
It is q ∈ R to be provided as motion vectors of the p for the origin of coordinates during rigid body displacement3, rigid body around p make rotate to be R ∈ SO
(3) a pair of displacement coordinates (q, R), then we can combine both, are obtained to represent this rigid body displacement, it is all such
Coordinate constitutes a set SE (3)：SE (3)={ (q, R)：q∈R3, R ∈ SO (3) } and=R3×SO(3).Make x1, x2Difference table
Show the position before translation and after translation, then can obtain：
x2=q+Rx1, wherein q, R ∈ SE (3) represent the displacement of rigid body, and the displacement of the element representation rigid body in SE (3) becomes
Change.In order to which translation and rotation are showed with simpler clearly method, the conversion of rigid body is represented using homogeneous matrix.
During homogeneous partial differential, position a little is represented with 1 here, vector is represented with 0.
The evolution of one point can be expressed as：
It is g ∈ SE (3) homogeneous expression for 4 × 4 matrixes
SE (3) is redefined, the whole homogeneous partial differentials of element in former SE (3) new SE (3) is obtained into, new SE (3) is multiplying
A group can be constituted under method computing, SE (3) meets rigid body displacement condition, therefore SE (3) can represent the displacement of rigid body,Represent
Be exactly rotation transformation around axle, q be in rotary shaft a bit, because coordinate system is therefore the position using the q points before displacement as origin
Q points after shifting have reflected the translation transformation situation of q points, and SE (3) is a Lie group.There will necessarily be a Lie algebra can reflect SE
(3) full detail.
The human behavior feature based on 3D bones is described using SE (3).The object of Activity recognition be video or image sequence,
In these objects, the action of people is all discrete, that is to say, that the motion of people is arranged by several static frame configurations
Row are formed, therefore it is exactly to find that the side of static skeletal shape can be described that another characteristic description is known in the human behavior based on 3D bones
Method.Position relationship between two sections of bones is interpreted as displacement mapping relations, that is to say, that one section of bone is regarded as another section
The result of bone displacement, in theorem in Euclid space, a certain section of fixed bone equally can also be used relative to the position of another section of bone
Lie group shows, and then finds corresponding Lie algebra as motion characteristics.The purpose of the relative description method of Lie algebra is exactly
The relative position relation between each bone and other bones is calculated, and is represented using ve (B).Assuming that a skeleton model has
N sections of bone rigid bodies, then in t, the shape of this skeleton model can be expressed as C (t)=(ve (B1,2), ve
(B1,3) ..., ve (BN, n-1)), shared quantity 6 × n × (n-1) of this vector is individual.
The process object of dynamic time warping is the ordered series of numbers arranged in time series, makes the two arrays respectively X：
x1, x2, x3..., xNAnd Y：y1, y2, y3..., yM, length is respectively N, M ∈ R.The two arrays are probably discrete signal, or
Person is more common, is the time equidistant points produced during acquisition characteristics ordered series of numbers.Here this feature space is set as F, that
Xn, ym∈ F, wherein n ∈ [1, N], m ∈ [1, M].Generally, when comparing two different feature x, y ∈ F, it is necessary to a function
As the method for comparing two characteristic similarities, this function meets c：F×F→R≥0.If x and y similitude is higher, c
The value of (x, y) is just smaller.Consider that the element in all X and Y is constituted several right, the purpose of dynamic time warping is exactly to find one
Optimal combination, make all c (x, y) for being obtained after the element combination of two in X and Y be added after value it is minimum.
Dynamic time warping has in itself can not handle the defect of the different ordered series of numbers of dimension.The present invention is using a kind of improved
Method, integrates depth typical time period regular.And canonical correlation analysis is introduced, strengthen the similar of two ordered series of numbers that needs align
Property, in the case where similarity is higher, the accuracy rate of alignment, which can have, significantly to be lifted.Correlation between two data is object
Between linear connection measurement.Two groups of higher data of correlation are more similar.The correlation reflection of two groups of data be this two
The group linear degree of correlation of data, correlation is bigger, the easier distribution for going out another group of data from one group of data prediction.In behavior
During identification, the frame configuration of two high array representations of the degree of correlation also has higher similarity.Canonical correlation analysis
Effect be exactly that two groups of data are made with the higher degree of correlation by linear transformation.
Linear transformation in canonical correlation analysis is changed into nonlinear transformation by depth canonical correlation analysis, makes two numbers
The similarity of row after transformation reaches higher level.Deep learning, which refers to, has more than two layers of depth deep layer network, Neng Goutong
The nonlinear function for crossing abstract multilayer nest formation is obtained closer to real effect., can be by initial by deep learning
Change parameter and by training the form updated to obtain good effect.In depth canonical correlation analysis, deep learning is main
It is to find the nonlinear transformation for making the degree of correlation of two ordered series of numbers higher.
Ordered series of numbers in deep learning passes through after nonlinear transformation, and the loss letter of deep layer network is calculated by corresponding method
Number, the gradient of each weight is then determined according to loss function, weight is updated on the basis of gradient, is finally reached stable shape
State, weight now is exactly the final result after deep layer network updates repeatedly, and last output result is exactly the solution of deep layer network.
Regular typical time period is before alignment ordered series of numbers, ordered series of numbers to be carried out into canonical correlation analysis first, this two groups are improved
The correlation of data, in the case of high similitude, the alignment of ordered series of numbers can obtain more preferable effect.
Regular depth typical time period is to carry out nonlinear transformation, Ran Houjin first with the function pair data in deep layer network
Row canonical correlation analysis, the parameters in Nonlinear Mapping are updated by loss function, finally give stable result.
Fusion depth typical time period is regular to be mainly be improved regular to depth typical time period.It is specific to improve deep layer network
In loss function calculation.To the sample f Jing Guo nonlinear transformation1, f2, f3..., calculated using dynamic time warping
Process obtains the function f of an integrationdtw, then we select the f in sample1, calculated using the two samples
In counting loss function, f is utilizeddtwAnd f1The U and V obtained after calculating covariance, K and singular value decomposition K
All it is to be calculated by the two samples, by each sample f1, f2, f3... is calculated as
Only need to calculate a K and SVD decomposition in a circulation of deep layer network, simultaneouslyAndSimilarly only need to calculate once, the substantial amounts of time can be saved.
In order to improve the accuracy of identification, we attempt to be modified feature using correlation.Pass through the survey after alignment
The correlation for the general characteristic that sample sheet and training are obtained, the effect of test data alignment.Due to LARP motion features
The structure of matrix represents that Fig. 9 calculates for corresponding coefficient correlation as shown in figure 8, being characterized time series in the structure of matrix
Mode.The effect alignd is assessed by the corresponding coefficient between each row of test matrix row corresponding with general matrix.
By the use of coefficient correlation as the assessment of test sample correlation, work of the higher frame number of increase correlation in identification
With the effect of the relatively low frame number of reduction correlation.To distinguish the difference between correlation, coefficient correlation x ∈ R are assign as variograph
Coefficient is calculated, the weight of frame number is changed by the form of multiplication, selection correction function is ex, amendment flow such as Figure 10.By integrating
The regular adjustment to sample correlations of depth time, correlation between test sample and training result it is most of 0-0.2 it
Between, it is few negatively correlated situation occur.Therefore, it is possible to distinguish the frame that the relatively low frame of correlation is high with other correlations, and reduce
The low frame weight of correlation.
Beneficial effects of the present invention：
Effect one, by using merging, depth typical time period is regular and Relative modification alignment improves recognition accuracy；
Effect two, overcomes the defect of existing 3D bones Activity recognition；
Effect three, uses the fusion depth typical time period regular calculating time for significantly reducing Activity recognition.
Brief description of the drawings
The present invention is further described with reference to the accompanying drawings and examples.
3D Activity recognition schematic flow sheets in Fig. 1, embodiment 1.
Fig. 2, the Activity recognition result schematic diagram of embodiment 1.
Fig. 3, integrates the regular deep layer schematic network structure of depth typical time period.
Fig. 4, dynamic time warping integration method schematic diagram.
Fig. 5, rigid motion splits schematic diagram.
Fig. 6, human motion behavior bone schematic diagram.
Fig. 7, correction function schematic diagram.
Fig. 8, the time series of matrix represents schematic diagram.
Fig. 9, coefficient correlation calculates schematic diagram.
Figure 10, corrects schematic flow sheet.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, with reference to embodiments, to the present invention
It is further elaborated.It should be appreciated that specific embodiment described herein is not used to limit only to explain the present invention
The fixed present invention.
Embodiment 1
The present embodiment provide it is a kind of based on integrating, depth typical time period is regular and Activity recognition method of related amendment, including
Image preprocessing, graphical analysis, image understanding, such as Fig. 1, described image analysis include：
(1) it is rigid body displacement by the behavior representation of human body, rigid body displacement is decomposed into translation of rigid body and rigid body rotates, with neat
Submatrix Lie group SE (3) represents rigid body displacement, Lie algebra SO (3) reflection homogeneous matrix Lie group SE (3) full details；
(2) skeleton data of collection, which is set up in skeleton model C (t), skeleton model C (t), N sections of bone rigid bodies, in t
Carve, one section of bone is defined as to the result of another section of bone displacement, the position relationship between one section of bone and another section of bone is determined
Justice is displacement mapping relations, and in theorem in Euclid space, displacement mapping relations are expressed as into homogeneous matrix Lie group SE (3), then will be neat
The corresponding Lie algebra SO (3) of submatrix Lie group SE (3) are as motion feature, and foundation describes method based on Lie algebra relative characteristic
Skeleton model C (t), and difference processing is carried out to skeleton model C (t),
Wherein, N is positive integer；
(3) using the depth typical time period regular method alignment skeleton model C (t), the feature samples after being alignd；
(4) alignment feature sample in correlative character, amendment step (3) is utilized, revised alignment feature sample is obtained；
(5) revised feature samples are classified using SVMs, obtains Activity recognition result.
The present embodiment uses the Florence3D-Action data sets made by Kinect to be used as experimental subjects.
Kinect includes 3 cameras altogether, and middle is RGB cameras, for catching the coloured image that resolution ratio is 640 × 480,
Obtain 30 two field pictures each second, both sides are two depth transducers, for detecting the relative position of user.Florence3D-
The Kinect sensor that Action data sets are fixed using a position collects data.It carries out 9 respectively by 10 different people
The different actions planted are constituted.Everyone act two to three times each time, whole a total of 215 actions of data set.Data
The skeleton pattern of concentration is made up of 15 joints.In the present embodiment, the action of half is selected everyone as training sample, will
Remaining half is used as test sample.
Framework information in Florence3D-Action data sets is processed on the skeleton exported in Kinect platforms
Arrive.The speed of Kinect video is maintained at 30 frames.In Kinect skeleton pattern, such as Fig. 6, is behavior bone schematic diagram,
Similarly, forearm, upper arm, trunk, these body parts of head are considered as rigid body.Set up an absolute coordinate system and represent each
Joint, origin is Kinect position, and x-axis is horizontal left direction, and vertically upward, z-axis is to often immediately ahead of Kinect to y-axis
There is a position coordinates in individual joint in a coordinate system.Correlation during in view of joint motions, Kinect archetype
Be present substantial amounts of redundancy in data, it is therefore desirable to be improved on the basis of these data.Some in human body are assumed in this implementation
The higher joint of correlation is relatively fixed constant, and the trunk of human body is a larger rigid body.Joint on skeleton just by
It divide into two-stage：One-level is the artis on trunk；Two grades are to be located at the outer son by bone being connected with one-level joint of trunk to close
Node.
In order to determine the position in one-level joint, model is set up coordinate system using the vertebra tail position of human body as origin, obtained
Coordinate (u, r, t)0, (u, r, t)1, (u, r, t)2(u, r, t)3.Using each one-level as coordinate origin, coordinate system is set up respectively
Obtain two grades of body joint point coordinates.
The workflow of the present embodiment：Bone parameters model is gathered, bone parameters are described using LARP methods, result is made
Alignd with depth typical time period is regular, then carry out Relative modification, classification output finally is carried out to result.
The regular method of depth typical time period is regular for the typical time period for introducing deep learning.When introducing the typical case of deep learning
Between it is regular be the activation primitive using weight W (t) and deep layer network to skeleton model C (t) progress nonlinear transformations, converted
Sample C, (t) converts sample C, first group of data C in (t), and 1, as normal data, will be converted using dynamic time warping method
Sample C, (t) respectively with normal data C, 1 alignment obtains the sample C that aligns, (t), according to alignment sample C, (t) and normal data
C ' 1 and：
Gradient G (t) corresponding with the sample C ' (t ') that aligns is calculated, is updated according to the learning rate of gradient G (t) and deep layer network
Weight W (t), is repeated until meeting deep layer network convergence condition；By the training characteristics C of all deep layer e-learnings1f, C2f,
C3f...-and C1fIt is average after being alignd with dynamic time warping method, calculate training result
Wherein, total number of plies of deep layer network is at least 2.
The regular method of depth typical time period is specially：The original skeleton motion data X of input；Obtain LARP features C1, C2,
C3…；Utilize weight W1, W2, W3Activation primitive with network is to C1, C2, C3…-Nonlinear transformation is carried out, C ' is obtained1, C '2, C
′3…；
By C '1As template, C ' is made1, C '2, C '3... respectively with C '1C " is obtained using DTW alignment1, C "2, C "3…；C″1,
C″2, C "3... according to gradient formula respectively with C '1Obtain corresponding gradient G1, G2, G3….；
Gradient formula is：
According to G1, G2, G3... and the learning rate of network settings updates weight W1, W2, W3…；
Preferably, it is the reduction calculating time.Regular depth typical time period is preferably the fusion regular method of depth typical time period
Alignd.
Wherein, amendment flow such as Figure 10.The correlative character include by skeleton model C (t) with align after feature sample
This alignment, calculates the correlation of skeleton model C (t) and each frame of feature samples after aliging, by the use of coefficient correlation x ∈ R as
Variable calculates correction factor, the weight for the corresponding frame of change that correction factor and the characteristic of each frame are multiplied, correction factor clothes
From correction function ex.Wherein test sample is skeleton model, and the feature samples after alignment are training result.
Because the structure of LARP motion feature matrixes is as shown in figure 8, each row and generality that pass through such as Fig. 9 test matrixs
Corresponding coefficient between matrix is arranged accordingly assesses the effect alignd.
Preferably, in order to reduce the time-consuming of alignment algorithm, depth typical time period is regular can be optimized for integrating depth typical case
Time alignment.Therefore, step (3) the regular method of depth typical time period of integrating includes：Utilize weight W (t) and deep layer network
Activation primitive nonlinear transformation is carried out to skeleton model C (t), obtain conversion sample C ' (t), will be the in conversion sample C ' (t)
One group of data C ' 1 will convert sample C as normal data using dynamic time warping method, (t) respectively with normal data C ' 1
Alignment, be averaging after obtain integration characteristics C, according to closing feature C and normal data C, 1 and：
Calculate covariance, and carry out SVD and decompose to obtain U, V, according to U, V, conversion sample C, (t) and
Calculate with converting the one-to-one gradient Gs (t) of sample C ' (t), according to the study of gradient G (t) and deep layer network
Rate updates convergence number of times in weight W (t), the present embodiment and is manually set to 5 times, after convergence, by all process deep layer network transformations
Training characteristics C1f, C2f, C3f…-Training characteristics C corresponding with normal data C ' 11fIt is average after being alignd with dynamic time warping,
Result of calculation is alignment feature sample.
Specifically, step (2) the difference processing is including the use of interpolation method, and the interpolation method includes：
Frame number identical feature is obtained by interpolation, interpolation formula is：
Wherein,Define Q1, Q2, Q3…Qn∈ SE (3) are respectively Jt1, t2, t3…tnMoment
Instantaneous relative position.
Wherein, skeleton model C (t) bags that method is described based on Lie algebra relative characteristic are set up described in the step (2)
Include：Select the e in skeleton modelmBone and enBone is analyzed, emThe end points of bone is em1And em2, enThe end points of bone is
en1And en2, with end points en1For origin, enThe direction vector of bone is axle, bone emWith bone enResiding face is that coordinate surface sets up seat
Mark system, emThe end points e of bonem1And em2Relative position be expressed as：
enEnd points relative to emPosition be：
emBone and enThe displacement mapping relations of bone are emBone and enThe relative position relation of bone is：
According to emBone and enThe relative position relation of bone calculates each bone and the phase shift mapping relations of other bones
Ve (B), defining skeleton model has n sections of bone rigid bodies, show that the skeleton model of t is shaped as C (t) and is：
C (t)=(ve (B1,2), ve (B1,3) ..., ve (BN, n-1)),
Ve (B)=(u1, u2, u3, v1, v2, v3)
Wherein, C (t) has 6 × n × (n-1) individual vector, and n and m is the positive integer less than N.
Preferably, described image pretreatment also includes image noise reduction and image enhaucament.Recognition effect can be improved.
Depth typical time period ordered structure such as Fig. 3 is integrated in embodiment 1, wherein, the deep learning deep layer net that we use
Network is BP network structures.4 layers, including input layer, output layer and two hidden layers are provided with deep layer network altogether.Such as Fig. 5, Lie group is used
The displacement of rigid body can be represented, the shift transformation of rigid body can resolve into translation and rotation two parts.
During training, dynamic time warping method, i.e. DTW, flow such as Fig. 4 be：Training data is selected to concentrate
First array as aliging with reference to the data all to other, then averagely obtain a new reference array, general
After the step is circulated 25 times, integral data is finally given.Because deep layer network only needs to calculate a subgradient, compare and change
DCTW before entering, our deep layer network has improvement on the time of algorithm, and the algorithm time is as shown in table 1 below.
Table 1
During Relative modification is carried out to feature, amendment purpose is to distinguish the frame of video of good relationship
The poor frame of video with correlation, the requirement to function is to expand the influence that correlation is produced to correction factor, derivative as far as possible
Larger function can produce preferable effect in our experiment.The best function of selection result is used as our amendment letter
Number, selection accuracy rate highest exIt is used as correction function, exFunctional image on the interval of [- 1,1] is as shown in Figure 7.
Correction function data are modified after result such as table 2.Final Relative modification result is compared to most
First LARP+DTW has 0.62% raising.
Table 2
| Experiment | As a result |
| LARP+DTW | 0.9071 |
| LARP+DCTW | 0.9124 |
| LARP+DCTW+ Relative modifications | 0.9133 |
After the amendment of correlation, Activity recognition result such as Fig. 2.Activity recognition method in the present embodiment is to some
The relatively low action of discrimination improves more obvious, and the two actions of for example drinking water and make a phone call, the accuracy rate of identification is all improved
1%-3% or so.
Although illustrative embodiment of the invention is described above, in order to the technology of the art
Personnel are it will be appreciated that the present invention, but the present invention is not limited only to the scope of embodiment, to the common skill of the art
For art personnel, as long as long as various change is in the spirit and scope of the invention that appended claim is limited and is determined, one
The innovation and creation using present inventive concept are cut in the row of protection.
Claims (8)
1. a kind of, based on integrating, depth typical time period is regular and Activity recognition method of related amendment, including image preprocessing, figure
As analysis, image understanding, it is characterised in that：Described image analysis includes：
(1) it is rigid body displacement by the behavior representation of human body, rigid body displacement is decomposed into translation of rigid body and rigid body rotates, homogeneous square is used
Battle array Lie group SE (3) represents rigid body displacement, Lie algebra SO (3) reflection homogeneous matrix Lie group SE (3) full details；
(2) skeleton data of collection, which is set up in skeleton model C (t), skeleton model C (t), N sections of bone rigid bodies, in t, will
One section of bone is defined as the result of another section of bone displacement, and the position relationship between one section of bone and another section of bone is defined as position
Mapping relations are moved, in theorem in Euclid space, displacement mapping relations homogeneous matrix Lie group SE (3) are expressed as, then by homogeneous matrix
The corresponding Lie algebra SO (3) of Lie group SE (3) set up the bone mould that method is described based on Lie algebra relative characteristic as motion feature
Type C (t), and difference processing is carried out to skeleton model C (t),
Wherein, N is positive integer；
(3) using the depth typical time period regular method alignment skeleton model C (t), the feature samples after being alignd；
(4) alignment feature sample in correlative character, amendment step (3) is utilized, revised alignment feature sample is obtained；
(5) revised feature samples are classified using SVMs, obtains Activity recognition result.
2. according to claim 1, based on integrating, depth typical time period is regular and Activity recognition method of related amendment, its
It is characterised by：Step (4) described correlative character includes aliging skeleton model C (t) with the feature samples after aliging, and calculates bone
Bone MODEL C (t) with align after each frame of feature samples correlation, using coefficient correlation x ∈ R be variable calculate amendment system
Number, the weight for the corresponding frame of change that correction factor and the characteristic of each frame are multiplied, correction factor obeys correction function ex。
3. according to claim 1 or 2, based on integrating, depth typical time period is regular and Activity recognition method of related amendment,
It is characterized in that：The regular method of depth typical time period described in step (3) is the integration regular method of depth typical time period.
4. based on integrating, depth typical time period is regular and Activity recognition method of related amendment according to claim 3, it is special
Levy and be：Step (3) the regular method of depth typical time period of integrating includes：
Nonlinear transformation is carried out to skeleton model C (t) using the activation primitive of weight W (t) and deep layer network, obtains converting sample
C ' (t), first group of data C ' 1 in conversion sample C ' (t), as normal data, is varied change using dynamic time warping method
This C ' (t) aligns with normal data C ' 1 respectively, be averaging after obtain integration characteristics C, according to closing feature C and normal data C ' 1
And：
<mrow>
<msub>
<mi>&Sigma;</mi>
<mn>11</mn>
</msub>
<mo>=</mo>
<mfrac>
<mn>1</mn>
<mrow>
<mi>T</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
</mfrac>
<msub>
<mi>f</mi>
<mn>1</mn>
</msub>
<msubsup>
<mi>Cf</mi>
<mn>1</mn>
<mi>T</mi>
</msubsup>
</mrow>
<mrow>
<msub>
<mi>&Sigma;</mi>
<mn>12</mn>
</msub>
<mo>=</mo>
<mfrac>
<mn>1</mn>
<mrow>
<mi>T</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
</mfrac>
<msub>
<mi>f</mi>
<mn>1</mn>
</msub>
<msubsup>
<mi>Cf</mi>
<mrow>
<mi>d</mi>
<mi>t</mi>
<mi>w</mi>
</mrow>
<mi>T</mi>
</msubsup>
</mrow>
<mrow>
<msub>
<mi>&Sigma;</mi>
<mn>22</mn>
</msub>
<mo>=</mo>
<mfrac>
<mn>1</mn>
<mrow>
<mi>T</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
</mfrac>
<msub>
<mi>f</mi>
<mrow>
<mi>d</mi>
<mi>t</mi>
<mi>w</mi>
</mrow>
</msub>
<msubsup>
<mi>Cf</mi>
<mrow>
<mi>d</mi>
<mi>t</mi>
<mi>w</mi>
</mrow>
<mi>T</mi>
</msubsup>
</mrow>
Calculate covariance, and carry out SVD and decompose to obtain U, V, according to U, V, conversion sample C ' (t) and
<mrow>
<msub>
<mi>F</mi>
<mn>1</mn>
</msub>
<mo>=</mo>
<msubsup>
<mi>&Sigma;</mi>
<mn>11</mn>
<mrow>
<mo>-</mo>
<mn>1</mn>
<mo>/</mo>
<mn>2</mn>
</mrow>
</msubsup>
<msup>
<mi>UV</mi>
<mi>T</mi>
</msup>
<msubsup>
<mi>&Sigma;</mi>
<mn>22</mn>
<mrow>
<mo>-</mo>
<mn>1</mn>
<mo>/</mo>
<mn>2</mn>
</mrow>
</msubsup>
<msub>
<mi>f</mi>
<mn>1</mn>
</msub>
</mrow>
<mrow>
<msub>
<mi>F</mi>
<mn>2</mn>
</msub>
<mo>=</mo>
<msubsup>
<mi>&Sigma;</mi>
<mn>11</mn>
<mrow>
<mo>-</mo>
<mn>1</mn>
<mo>/</mo>
<mn>2</mn>
</mrow>
</msubsup>
<msup>
<mi>USU</mi>
<mi>T</mi>
</msup>
<msubsup>
<mi>&Sigma;</mi>
<mn>11</mn>
<mrow>
<mo>-</mo>
<mn>1</mn>
<mo>/</mo>
<mn>2</mn>
</mrow>
</msubsup>
<msub>
<mi>f</mi>
<mi>i</mi>
</msub>
</mrow>
<mrow>
<mfrac>
<mrow>
<mo>&part;</mo>
<mo>|</mo>
<mo>|</mo>
<mi>K</mi>
<mo>|</mo>
<msub>
<mo>|</mo>
<mo>*</mo>
</msub>
</mrow>
<mrow>
<mo>&part;</mo>
<msub>
<mi>f</mi>
<mn>1</mn>
</msub>
<mrow>
<mo>(</mo>
<msub>
<mi>X</mi>
<mn>1</mn>
</msub>
<mo>;</mo>
<msub>
<mi>&theta;</mi>
<mn>1</mn>
</msub>
<mo>)</mo>
</mrow>
</mrow>
</mfrac>
<mo>=</mo>
<mfrac>
<mn>1</mn>
<mrow>
<mi>T</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
</mfrac>
<mrow>
<mo>(</mo>
<msub>
<mi>F</mi>
<mn>1</mn>
</msub>
<mo>-</mo>
<msub>
<mi>F</mi>
<mn>2</mn>
</msub>
<mo>)</mo>
</mrow>
</mrow>
Calculate with converting the one-to-one gradient Gs (t) of sample C ' (t), according to the learning rate of gradient G (t) and deep layer network more
New weight W (t), after convergence, by the training characteristics C of all process deep layer network transformations1f, C2f, C3f…-With 1 couple of normal data C '
The training characteristics C answered1fAverage after being alignd with dynamic time warping, result of calculation is alignment feature sample；
Wherein, total number of plies of deep layer network is at least 2.
5. according to claim 4, based on integrating, depth typical time period is regular and Activity recognition method of related amendment, its
It is characterised by：The deep layer network is BP networks, and BP networks include an input layer, an output layer and two hidden layers.
6. according to claim 1 or 2, based on integrating, depth typical time period is regular and Activity recognition method of related amendment,
It is characterized in that：Step (2) the difference processing is including the use of interpolation method, and the interpolation method includes：Obtained by interpolation
Frame number identical feature, interpolation formula is：
<mrow>
<mi>&gamma;</mi>
<mrow>
<mo>(</mo>
<mi>t</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<msub>
<mi>Q</mi>
<mi>i</mi>
</msub>
<msub>
<mi>exp</mi>
<mrow>
<mi>S</mi>
<mi>E</mi>
<mrow>
<mo>(</mo>
<mn>3</mn>
<mo>)</mo>
</mrow>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mfrac>
<mrow>
<mi>t</mi>
<mo>-</mo>
<msub>
<mi>t</mi>
<mi>i</mi>
</msub>
</mrow>
<mrow>
<msub>
<mi>t</mi>
<mrow>
<mi>i</mi>
<mo>+</mo>
<mn>1</mn>
</mrow>
</msub>
<mo>-</mo>
<msub>
<mi>t</mi>
<mi>i</mi>
</msub>
</mrow>
</mfrac>
<msub>
<mi>B</mi>
<mi>i</mi>
</msub>
<mo>)</mo>
</mrow>
<mo>,</mo>
<mi>t</mi>
<mo>&Element;</mo>
<mo>&lsqb;</mo>
<msub>
<mi>t</mi>
<mi>i</mi>
</msub>
<mo>,</mo>
<msub>
<mi>t</mi>
<mrow>
<mi>i</mi>
<mo>+</mo>
<mn>1</mn>
</mrow>
</msub>
<mo>&rsqb;</mo>
<mo>,</mo>
</mrow>
Wherein,Define Q1, Q2, Q2...Qn∈ SE (3) are respectively t1, t2, t3...tnMoment
Instantaneous relative position.
7. according to based on integrating, depth typical time period is regular and Activity recognition method of related amendment described in claim 1 or 2, its
It is characterised by：Setting up the skeleton model C (t) for describing method based on Lie algebra relative characteristic described in the step (2) includes：Choosing
Select the e in skeleton modelmBone and enBone is analyzed, emThe end points of bone is em1And em2, enThe end points of bone is en1And
en2, with end points en1For origin, enThe direction vector of bone is axle, bone emWith bone enResiding face is that coordinate surface sets up coordinate
System, emThe end points e of bonem1And em2Relative position be expressed as：
<mrow>
<mfenced open = "[" close = "]">
<mtable>
<mtr>
<mtd>
<mrow>
<msubsup>
<mi>e</mi>
<mrow>
<mi>m</mi>
<mn>1</mn>
</mrow>
<mi>n</mi>
</msubsup>
<mrow>
<mo>(</mo>
<mi>t</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
<mtd>
<mrow>
<msubsup>
<mi>e</mi>
<mrow>
<mi>m</mi>
<mn>2</mn>
</mrow>
<mi>n</mi>
</msubsup>
<mrow>
<mo>(</mo>
<mi>t</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>1</mn>
</mtd>
<mtd>
<mn>1</mn>
</mtd>
</mtr>
</mtable>
</mfenced>
<mo>=</mo>
<mfenced open = "[" close = "]">
<mtable>
<mtr>
<mtd>
<mrow>
<msub>
<mi>R</mi>
<mrow>
<mi>m</mi>
<mo>,</mo>
<mi>n</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>t</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
<mtd>
<mrow>
<msub>
<mi>d</mi>
<mrow>
<mi>m</mi>
<mo>,</mo>
<mi>n</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>t</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>1</mn>
</mtd>
</mtr>
</mtable>
</mfenced>
<mfenced open = "[" close = "]">
<mtable>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<msub>
<mi>l</mi>
<mi>m</mi>
</msub>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>1</mn>
</mtd>
<mtd>
<mn>1</mn>
</mtd>
</mtr>
</mtable>
</mfenced>
</mrow>
enEnd points relative to emPosition be：
<mrow>
<mfenced open = "[" close = "]">
<mtable>
<mtr>
<mtd>
<mrow>
<msubsup>
<mi>e</mi>
<mrow>
<mi>n</mi>
<mn>1</mn>
</mrow>
<mi>m</mi>
</msubsup>
<mrow>
<mo>(</mo>
<mi>t</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
<mtd>
<mrow>
<msubsup>
<mi>e</mi>
<mrow>
<mi>n</mi>
<mn>2</mn>
</mrow>
<mi>m</mi>
</msubsup>
<mrow>
<mo>(</mo>
<mi>t</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>1</mn>
</mtd>
<mtd>
<mn>1</mn>
</mtd>
</mtr>
</mtable>
</mfenced>
<mo>=</mo>
<mfenced open = "[" close = "]">
<mtable>
<mtr>
<mtd>
<mrow>
<msub>
<mi>R</mi>
<mrow>
<mi>n</mi>
<mo>,</mo>
<mi>m</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>t</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
<mtd>
<mrow>
<msub>
<mi>d</mi>
<mrow>
<mi>n</mi>
<mo>,</mo>
<mi>m</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>t</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>1</mn>
</mtd>
</mtr>
</mtable>
</mfenced>
<mfenced open = "[" close = "]">
<mtable>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<msub>
<mi>l</mi>
<mi>n</mi>
</msub>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>1</mn>
</mtd>
<mtd>
<mn>1</mn>
</mtd>
</mtr>
</mtable>
</mfenced>
</mrow>
emBone and enThe displacement mapping relations of bone are emBone and enThe relative position relation of bone is：
<mrow>
<msub>
<mi>P</mi>
<mrow>
<mi>m</mi>
<mo>,</mo>
<mi>n</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>t</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mfenced open = "[" close = "]">
<mtable>
<mtr>
<mtd>
<mrow>
<msub>
<mi>R</mi>
<mrow>
<mi>m</mi>
<mo>,</mo>
<mi>n</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>t</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
<mtd>
<mrow>
<msub>
<mi>d</mi>
<mrow>
<mi>m</mi>
<mo>,</mo>
<mi>n</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>t</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>1</mn>
</mtd>
</mtr>
</mtable>
</mfenced>
</mrow>
<mrow>
<msub>
<mi>P</mi>
<mrow>
<mi>n</mi>
<mo>,</mo>
<mi>m</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>t</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mfenced open = "[" close = "]">
<mtable>
<mtr>
<mtd>
<mrow>
<msub>
<mi>R</mi>
<mrow>
<mi>n</mi>
<mo>,</mo>
<mi>m</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>t</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
<mtd>
<mrow>
<msub>
<mi>d</mi>
<mrow>
<mi>n</mi>
<mo>,</mo>
<mi>m</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>t</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>1</mn>
</mtd>
</mtr>
</mtable>
</mfenced>
<mo>;</mo>
</mrow>
According to emBone and enThe relative position relation of bone calculates each bone and the phase shift mapping relations ve of other bones
(B), defining skeleton model has n sections of bone rigid bodies, show that the skeleton model of t is shaped as C (t) and is：
C (t)=(ve (B1,2), ve (B1,3) ..., ve (BN, n-1)),
<mrow>
<mi>B</mi>
<mo>=</mo>
<mfenced open = "[" close = "]">
<mtable>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mrow>
<mo>-</mo>
<msub>
<mi>u</mi>
<mn>3</mn>
</msub>
</mrow>
</mtd>
<mtd>
<msub>
<mi>u</mi>
<mn>2</mn>
</msub>
</mtd>
<mtd>
<msub>
<mi>v</mi>
<mn>1</mn>
</msub>
</mtd>
</mtr>
<mtr>
<mtd>
<msub>
<mi>u</mi>
<mn>3</mn>
</msub>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<msub>
<mi>u</mi>
<mn>1</mn>
</msub>
</mtd>
<mtd>
<msub>
<mi>v</mi>
<mn>2</mn>
</msub>
</mtd>
</mtr>
<mtr>
<mtd>
<mrow>
<mo>-</mo>
<msub>
<mi>u</mi>
<mn>2</mn>
</msub>
</mrow>
</mtd>
<mtd>
<msub>
<mi>u</mi>
<mn>1</mn>
</msub>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<msub>
<mi>v</mi>
<mn>3</mn>
</msub>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
</mtr>
</mtable>
</mfenced>
</mrow>
2
Ve (B)=(u1, u2, u3, v1, v2, v3)
Wherein, C (t) has 6 × n × (n-1) individual vector, and n and m is the positive integer less than N.
8. according to based on integrating, depth typical time period is regular and Activity recognition method of related amendment described in claim 1 or 2, its
It is characterised by：Described image pretreatment also includes image noise reduction and image enhaucament.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710425906.1A CN107229920B (en) | 2017-06-08 | 2017-06-08 | Behavior identification method based on integration depth typical time warping and related correction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710425906.1A CN107229920B (en) | 2017-06-08 | 2017-06-08 | Behavior identification method based on integration depth typical time warping and related correction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107229920A true CN107229920A (en) | 2017-10-03 |
| CN107229920B CN107229920B (en) | 2020-11-13 |
Family
ID=59936262
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710425906.1A Active CN107229920B (en) | 2017-06-08 | 2017-06-08 | Behavior identification method based on integration depth typical time warping and related correction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107229920B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108830206A (en) * | 2018-06-06 | 2018-11-16 | 成都邑教云信息技术有限公司 | A kind of course axis Internet Educational System |
| CN109614899A (en) * | 2018-11-29 | 2019-04-12 | 重庆邮电大学 | A kind of human motion recognition method based on Lie group feature and convolutional neural networks |
| CN109871750A (en) * | 2019-01-02 | 2019-06-11 | 东南大学 | A kind of gait recognition method based on skeleton drawing sequence variation joint repair |
| CN110197195A (en) * | 2019-04-15 | 2019-09-03 | 深圳大学 | A kind of novel deep layer network system and method towards Activity recognition |
| CN110472497A (en) * | 2019-07-08 | 2019-11-19 | 西安工程大学 | A kind of motion characteristic representation method merging rotation amount |
| CN111709323A (en) * | 2020-05-29 | 2020-09-25 | 重庆大学 | Gesture recognition method based on lie group and long-and-short term memory network |
| CN111832427A (en) * | 2020-06-22 | 2020-10-27 | 华中科技大学 | EEG classification transfer learning method and system based on Euclidean alignment and Procrustes analysis |
| CN117647788A (en) * | 2024-01-29 | 2024-03-05 | 北京清雷科技有限公司 | Dangerous behavior identification method and device based on human body 3D point cloud |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673403A (en) * | 2009-10-10 | 2010-03-17 | 安防制造（中国）有限公司 | Target following method in complex interference scene |
| CN103778227A (en) * | 2014-01-23 | 2014-05-07 | 西安电子科技大学 | Method for screening useful images from retrieved images |
| US20140169623A1 (en) * | 2012-12-19 | 2014-06-19 | Microsoft Corporation | Action recognition based on depth maps |
| CN105930767A (en) * | 2016-04-06 | 2016-09-07 | 南京华捷艾米软件科技有限公司 | Human body skeleton-based action recognition method |
| US20160378861A1 (en) * | 2012-09-28 | 2016-12-29 | Sri International | Real-time human-machine collaboration using big data driven augmented reality technologies |
| CN106384093A (en) * | 2016-09-13 | 2017-02-08 | 东北电力大学 | Human action recognition method based on noise reduction automatic encoder and particle filter |
-
2017
- 2017-06-08 CN CN201710425906.1A patent/CN107229920B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673403A (en) * | 2009-10-10 | 2010-03-17 | 安防制造（中国）有限公司 | Target following method in complex interference scene |
| US20160378861A1 (en) * | 2012-09-28 | 2016-12-29 | Sri International | Real-time human-machine collaboration using big data driven augmented reality technologies |
| US20140169623A1 (en) * | 2012-12-19 | 2014-06-19 | Microsoft Corporation | Action recognition based on depth maps |
| CN103778227A (en) * | 2014-01-23 | 2014-05-07 | 西安电子科技大学 | Method for screening useful images from retrieved images |
| CN105930767A (en) * | 2016-04-06 | 2016-09-07 | 南京华捷艾米软件科技有限公司 | Human body skeleton-based action recognition method |
| CN106384093A (en) * | 2016-09-13 | 2017-02-08 | 东北电力大学 | Human action recognition method based on noise reduction automatic encoder and particle filter |
Non-Patent Citations (3)
| Title |
|---|
| GEORGE TRIGEORGIS 等: "Deep Canonical Time Warping", 《2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION(CVPR)》 * |
| RAVITEJA VEMULAPALLI 等: "Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group", 《2014 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
| 景运革: "《基于粗糙集属性约简算法的研究》", 31 July 2013 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108830206A (en) * | 2018-06-06 | 2018-11-16 | 成都邑教云信息技术有限公司 | A kind of course axis Internet Educational System |
| CN109614899B (en) * | 2018-11-29 | 2022-07-01 | 重庆邮电大学 | Human body action recognition method based on lie group features and convolutional neural network |
| CN109614899A (en) * | 2018-11-29 | 2019-04-12 | 重庆邮电大学 | A kind of human motion recognition method based on Lie group feature and convolutional neural networks |
| CN109871750A (en) * | 2019-01-02 | 2019-06-11 | 东南大学 | A kind of gait recognition method based on skeleton drawing sequence variation joint repair |
| CN109871750B (en) * | 2019-01-02 | 2023-08-18 | 东南大学 | Gait recognition method based on skeleton diagram sequence abnormal joint repair |
| CN110197195A (en) * | 2019-04-15 | 2019-09-03 | 深圳大学 | A kind of novel deep layer network system and method towards Activity recognition |
| WO2020211243A1 (en) * | 2019-04-15 | 2020-10-22 | 深圳大学 | Behavior identification method and apparatus based on deep network technology, and storage medium |
| CN110197195B (en) * | 2019-04-15 | 2022-12-23 | 深圳大学 | Novel deep network system and method for behavior recognition |
| CN110472497A (en) * | 2019-07-08 | 2019-11-19 | 西安工程大学 | A kind of motion characteristic representation method merging rotation amount |
| CN111709323A (en) * | 2020-05-29 | 2020-09-25 | 重庆大学 | Gesture recognition method based on lie group and long-and-short term memory network |
| CN111709323B (en) * | 2020-05-29 | 2024-02-02 | 重庆大学 | Gesture recognition method based on Liqun and long-short-term memory network |
| CN111832427A (en) * | 2020-06-22 | 2020-10-27 | 华中科技大学 | EEG classification transfer learning method and system based on Euclidean alignment and Procrustes analysis |
| CN111832427B (en) * | 2020-06-22 | 2022-02-18 | 华中科技大学 | EEG classification transfer learning method and system based on Euclidean alignment and Procrustes analysis |
| CN117647788A (en) * | 2024-01-29 | 2024-03-05 | 北京清雷科技有限公司 | Dangerous behavior identification method and device based on human body 3D point cloud |
| CN117647788B (en) * | 2024-01-29 | 2024-04-26 | 北京清雷科技有限公司 | Dangerous behavior identification method and device based on human body 3D point cloud |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107229920B (en) | 2020-11-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107229920A (en) | Based on integrating, depth typical time period is regular and Activity recognition method of related amendment | |
| CN105787439B (en) | A kind of depth image human synovial localization method based on convolutional neural networks | |
| CN110135375B (en) | Multi-person attitude estimation method based on global information integration | |
| CN108932500B (en) | A kind of dynamic gesture identification method and system based on deep neural network | |
| CN111145131B (en) | Infrared and visible light image fusion method based on multiscale generation type countermeasure network | |
| CN106384093B (en) | A kind of human motion recognition method based on noise reduction autocoder and particle filter | |
| CN105095833B (en) | For the network establishing method of recognition of face, recognition methods and system | |
| CN107492121B (en) | Two-dimensional human body bone point positioning method of monocular depth video | |
| CN105160400A (en) | L21 norm based method for improving convolutional neural network generalization capability | |
| CN112069933A (en) | Skeletal muscle stress estimation method based on posture recognition and human body biomechanics | |
| CN105184767B (en) | A kind of movement human posture method for measuring similarity | |
| CN108764066A (en) | A kind of express delivery sorting working specification detection method based on deep learning | |
| CN111062340A (en) | Abnormal gait behavior identification method based on virtual posture sample synthesis | |
| CN109299701A (en) | Expand the face age estimation method that more ethnic group features cooperate with selection based on GAN | |
| CN106548194B (en) | The construction method and localization method of two dimensional image human joint points location model | |
| CN107657204A (en) | The construction method and facial expression recognizing method and system of deep layer network model | |
| CN111160294B (en) | Gait recognition method based on graph convolution network | |
| CN113128424B (en) | Method for identifying action of graph convolution neural network based on attention mechanism | |
| CN111199207B (en) | Two-dimensional multi-human body posture estimation method based on depth residual error neural network | |
| CN110135277B (en) | Human behavior recognition method based on convolutional neural network | |
| CN104298974A (en) | Human body behavior recognition method based on depth video sequence | |
| CN112750198B (en) | Dense correspondence prediction method based on non-rigid point cloud | |
| CN116012950B (en) | Skeleton action recognition method based on multi-heart space-time attention pattern convolution network | |
| CN111476077A (en) | Multi-view gait recognition method based on deep learning | |
| CN109919036A (en) | Worker's work posture classification method based on time-domain analysis depth network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20210111 Address after: 401120 room 701, room 1, 7 / F, building 11, No. 106, west section of Jinkai Avenue, Yubei District, Chongqing Patentee after: Chongqing Space Visual Creation Technology Co.,Ltd. Address before: 400044 No. 174 Sha Jie street, Shapingba District, Chongqing Patentee before: Chongqing University |
|
| TR01 | Transfer of patent right | ||
| CP03 | Change of name, title or address |
Address after: 401121 room 701, room 1, floor 7, building 11, No. 106, west section of Jinkai Avenue, Yubei District, Chongqing Patentee after: Space Shichuang (Chongqing) Technology Co.,Ltd. Address before: 401120 room 701, room 1, 7 / F, building 11, No. 106, west section of Jinkai Avenue, Yubei District, Chongqing Patentee before: Chongqing Space Visual Creation Technology Co.,Ltd. |
|
| CP03 | Change of name, title or address | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20231123 Address after: 401331 CQ-01-B2-XN020, Building 2, No. 37 Jingyang Road, Huxi Street, Shapingba District, Chongqing City Patentee after: Chongqing Space New Vision Artificial Intelligence Technology Research Institute Co.,Ltd. Address before: 401121 room 701, room 1, floor 7, building 11, No. 106, west section of Jinkai Avenue, Yubei District, Chongqing Patentee before: Space Shichuang (Chongqing) Technology Co.,Ltd. |
|
| TR01 | Transfer of patent right |