CN106228109A - A kind of action identification method based on skeleton motion track - Google Patents
A kind of action identification method based on skeleton motion track Download PDFInfo
- Publication number
- CN106228109A CN106228109A CN201610538399.8A CN201610538399A CN106228109A CN 106228109 A CN106228109 A CN 106228109A CN 201610538399 A CN201610538399 A CN 201610538399A CN 106228109 A CN106228109 A CN 106228109A
- Authority
- CN
- China
- Prior art keywords
- skeleton
- sequence
- convolutional neural
- neural networks
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
Abstract
The present invention relates to a kind of action identification method based on skeleton motion track, comprise: choose some human skeletal's Sequence composition training datasets comprising complete action, each human skeletal's sequence comprises the mankind's each skeleton Nodes Three-dimensional coordinate；Each human skeletal's sequence to training dataset, processes；Utilize convolutional neural networks, the skeleton trajectory diagram to three directions generated based on training dataset, does convolution respectively refreshing by the model training of network, respectively obtains a convolutional neural networks model；For test sample, after processing, each the test skeleton trajectory diagram that colouring information represents the temporal information of skeleton motion it is obtained by；By each test skeleton trajectory diagram, it is separately input in the convolutional neural networks model trained accordingly, it is possible to draw the classification results of convolutional neural networks；Obtain final classification results.The present invention can identify human action accurately and reliably.
Description
Technical field
The invention belongs to multimedia signal processing field, relate to a kind of action identification method.
Background technology
The motion detection recognition methods of the mankind, has application widely at society, such as: intelligent monitoring, people
The mutual somatic sensation television game of machine, video frequency searching etc..
The motion detection identification of the mankind, from based on traditional rgb video row transition to the most popular RGB-D video sequence
Row, movement locus as important feature growth always.The seizure of traditional movement locus is often based upon the detection of characteristic point
Algorithm, different feature point detecting methods can draw diverse movement locus.Simultaneously as characteristic point is in the inspection of different frame
Rope is highly unstable, and in whole video sequence, characteristic point is often discontinuous, therefore for feature point trajectory fado use based on
Histogrammic statistical method, after whole video sequence is calculated and added up, uses the grader such as support vector machine to carry out point
Class.
Human action based on three-dimensional skeleton motion track identifies kinds of schemes in recent years.But it is currently based on three-dimensional
The method that the movement locus sequence of skeleton carries out action recognition, is mostly and directly processes skeleton data, or convert thereof into Nogata
Figure is further processed again.
Summary of the invention
It is an object of the invention to provide the action identification method that a kind of discrimination based on skeleton motion track is higher, this
Bright is picture based on three-dimensional skeleton motion track by three-dimensional skeleton motion trajectory map, uses and has on picture classification remarkably
The convolutional neural networks of performance, carries out machine learning to the picture of three-dimensional skeleton motion trajectory map and classifies.Technical scheme is such as
Under:
A kind of action identification method based on skeleton motion track, comprises following step:
(1) some human skeletal's Sequence composition training datasets comprising complete action, each human skeletal's sequence are chosen
All comprise the mankind's each skeleton Nodes Three-dimensional coordinate；
(2) each human skeletal's sequence to training dataset, does following process:
1) for comprising the three-dimensional skeleton sequence of certain action of the mankind's each skeleton Nodes Three-dimensional coordinate, sequentially in time will
The three-dimensional coordinate position of skeleton node carries out line, obtains this skeleton node three-dimensional skeleton motion rail in whole skeleton sequence
Mark.
2) in cartesian coordinate system, the three-dimensional skeleton motion track of whole action is thrown on three coordinate planes respectively
Shadow, thus three-dimensional skeleton motion track is converted into three two-dimentional skeleton motion trajectory diagrams, it is defined as skeleton trajectory diagram；
3) on three skeleton trajectory diagrams, the skeleton orbit segment to different time, give different colors, the value of color
Depend on this skeleton orbit segment time location in whole skeleton sequence, thus be obtained by colouring information and represent skeleton motion
Each skeleton trajectory diagram of temporal information；
(3) convolutional neural networks is utilized, the skeleton trajectory diagram to three directions generated based on training dataset, does respectively
Convolution god, by the model training of network, respectively obtains a convolutional neural networks model；
(4) for test sample, after the process through step (2), it is obtained by colouring information and represents skeleton motion
Each of temporal information test skeleton trajectory diagram；
(5) by each test skeleton trajectory diagram, it is separately input in the convolutional neural networks model trained accordingly,
Can be derived that the classification results of convolutional neural networks, each convolutional neural networks can export an a length of training class number
Scores vector；
(6) three Scores vectors are sued for peace, and take the class label at extreme value place as final classification results.
Accompanying drawing explanation
Accompanying drawing 1 is whole human action's identification framework based on skeleton motion track
Detailed description of the invention
Following skeleton sequence is taken from what the data set such as MSRC-12, G3D, UTD-MHAD provided human skeletal's sequence.
1) assuming that complete human action's skeleton sequence contains n frame, represent with H, the human skeletal of each frame is by m
Individual node is constituted.Skeleton sequence H can use formula H={F1, F2..., FnRepresent.WhereinRepresent
The set of the three-dimensional coordinate of each skeleton node of the i-th frame,Represent the three-dimensional skeleton node location of jth skeleton node.Thus,
In this skeleton sequence, the three-dimensional skeleton motion track T of all skeleton nodes can represent by equation below:
T={T1, T2..., Ti..., Tn-1}
WhereinThe three-dimensional motion of all skeletons represented in sequence in certain adjacent two frame
Track, in setRepresent the three-dimensional skeleton motion track that kth skeleton node is obtained in adjacent two frames.
2) for each skeleton node at the three-dimensional skeleton motion track of adjacent two interframe, can be orthogonal Descartes
A line segment it is expressed as in system.Therefore it can project in the three of Descartes's rhombic system plane respectively.By whole skeleton
The all three-dimensional skeleton motion track of sequence is superimposed after all projecting on Descartes's orthogonal plane.Whole skeleton sequence
Row just can represent with the skeleton trajectory diagram of three projecting directions.Each skeleton trajectory diagram contains the institute in skeleton sequence
There is skeleton track distribution on this projecting direction, thus the spatial information of human action can be by fully in skeleton trajectory diagram
Description.
3) in order to describe action message more accurately, in skeleton trajectory diagram, the present invention uses and gives the most in the same time
The method of the color that three-dimensional skeleton motion track is different, is added thereto the temporal information of human motion.Concrete operations are as follows:
Initially setting up the color gradients striped of an a length of C, the change of its color can be by changing in HSV color model
H (colourity) realize.The present invention uses from dark blue to red color gradients striped, in color gradients striped vector, we
Another ClRepresenting in color gradients striped C, correspondence position is the color at l.Three-dimensional for any two interframe of any skeleton
Skeleton motion trackAccording to formula:L can be obtained, thus obtain the face that this three-dimensional skeleton motion track is corresponding
Color Cl.The physical significance of this operation is, specifies skeleton motion according to skeleton motion track position in whole skeleton sequence
The color of track, thus in skeleton trajectory diagram, the skeleton motion track of same time period has identical color, different time sections
Skeleton motion track distinguished.So far, skeleton trajectory diagram contains spatial information and the time of human action simultaneously
Information.
4) utilizing convolutional neural networks to the outstanding classification capacity of picture and high-level ability in feature extraction, the present invention will
The skeleton trajectory diagram of three different directions projections in cartesian coordinate system generated from training dataset skeleton sequence, the most defeated
Enter the training carrying out model in three convolutional neural networks.Convolutional neural networks structure in the present invention have employed the most well-known
AlexNet network structure.After having trained, train three convolutional neural networks models are preserved respectively.
5) for each for the skeleton sequence tested, according to the generating mode of skeleton trajectory diagram, it is possible to Descartes
Three skeleton trajectory diagrams are generated in rhombic system.According to the convolutional neural networks model trained, by same test skeleton sequence
Three corresponding skeleton trajectory diagrams, are separately input in corresponding convolutional neural networks model, it is possible to draw three Scores to
Amount.This vector length is training class number, and the value in vector is the normalization probability of subordinate correspondence classification.
6) the skeleton trajectory diagram of each skeleton sequence, through three Scores vectors that convolutional neural networks obtains, represents
What this skeleton sequence learnt through convolutional neural networks on three projecting directions and calculated is subordinated to different classes of probability,
In the present invention, directly the Scores vector to three directions carries out corresponding classification sum operation, finally takes belonging to maximum
Classification is as the sub-categories of this video sequence.
It is present invention experimental result explanation on the data sets such as MSRC-12, G3D, UTD-MHAD below:
In the Realization of Simulation of the enterprising line algorithm of Matlab-2013b platform, training dataset and test data are calculated
Collection skeleton trajectory diagram (being three directions).We use the most public degree of depth learning framework Caffe, at linux system
Under, by NvidiaGTXTITANX video card, accelerate the training of convolutional neural networks.The comprehensive part of last Scores exists
Complete on Matlab-2013b platform.
This method identifies in data set internationally recognized human action and tests, training set and test in data set
The method of salary distribution of collection uses the Cross Subject method of salary distribution.Test result is as follows: at the MSRC-12 number comprising 12 class actions
According on collection, it is thus achieved that the recognition accuracy of 93.12%；On the G3D data set comprising 20 class actions, it is thus achieved that 94.24%
Recognition accuracy；On the UTD-MHAD data set comprising 27 class actions, it is thus achieved that the recognition accuracy of 85.81%.
Claims (1)
1. an action identification method based on skeleton motion track, comprises following step:
(1) choosing some human skeletal's Sequence composition training datasets comprising complete action, each human skeletal's sequence is wrapped
Containing the mankind's each skeleton Nodes Three-dimensional coordinate；
(2) each human skeletal's sequence to training dataset, does following process:
1) for comprising the three-dimensional skeleton sequence of certain action of the mankind's each skeleton Nodes Three-dimensional coordinate, sequentially in time by skeleton
The three-dimensional coordinate position of node carries out line, obtains this skeleton node three-dimensional skeleton motion track in whole skeleton sequence.
2) in cartesian coordinate system, the three-dimensional skeleton motion track of whole action is projected on three coordinate planes respectively,
Thus three-dimensional skeleton motion track is converted into three two-dimentional skeleton motion trajectory diagrams, it is defined as skeleton trajectory diagram；
3) on three skeleton trajectory diagrams, the skeleton orbit segment to different time, give different colors, the value of color depends on
In this skeleton orbit segment time location in whole skeleton sequence, thus be obtained by colouring information represent skeleton motion time
Between each skeleton trajectory diagram of information；
(3) convolutional neural networks is utilized, the skeleton trajectory diagram to three directions generated based on training dataset, does convolution respectively
God, by the model training of network, respectively obtains a convolutional neural networks model；
(4) for test sample, after the process through step (2), be obtained by colouring information represent skeleton motion time
Between each of information test skeleton trajectory diagram；
(5) by each test skeleton trajectory diagram, it is separately input in the convolutional neural networks model trained accordingly, it is possible to
Show that the classification results of convolutional neural networks, each convolutional neural networks can export an a length of training class number
Scores vector；
(6) three Scores vectors are sued for peace, and take the class label at extreme value place as final classification results.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610538399.8A CN106228109A (en) | 2016-07-08 | 2016-07-08 | A kind of action identification method based on skeleton motion track |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610538399.8A CN106228109A (en) | 2016-07-08 | 2016-07-08 | A kind of action identification method based on skeleton motion track |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106228109A true CN106228109A (en) | 2016-12-14 |
Family
ID=57519497
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610538399.8A Pending CN106228109A (en) | 2016-07-08 | 2016-07-08 | A kind of action identification method based on skeleton motion track |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106228109A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108052896A (en) * | 2017-12-12 | 2018-05-18 | 广东省智能制造研究所 | Human bodys' response method based on convolutional neural networks and support vector machines |
| CN108597578A (en) * | 2018-04-27 | 2018-09-28 | 广东省智能制造研究所 | A kind of human motion appraisal procedure based on two-dimensional framework sequence |
| CN108846348A (en) * | 2018-06-07 | 2018-11-20 | 四川大学 | A kind of Human bodys' response method based on three-dimensional skeleton character |
| CN109584345A (en) * | 2018-11-12 | 2019-04-05 | 大连大学 | Human motion synthetic method based on convolutional neural networks |
| CN109670401A (en) * | 2018-11-15 | 2019-04-23 | 天津大学 | A kind of action identification method based on skeleton motion figure |
| CN110490034A (en) * | 2018-05-14 | 2019-11-22 | 欧姆龙株式会社 | Motion analysis device, action-analysing method, recording medium and motion analysis system |
| CN110728183A (en) * | 2019-09-09 | 2020-01-24 | 天津大学 | Human body action recognition method based on attention mechanism neural network |
| CN111223549A (en) * | 2019-12-30 | 2020-06-02 | 华东师范大学 | Mobile end system and method for disease prevention based on posture correction |
| CN112101262A (en) * | 2020-09-22 | 2020-12-18 | 中国科学技术大学 | Multi-feature fusion sign language recognition method and network model |
| CN112507940A (en) * | 2020-12-17 | 2021-03-16 | 华南理工大学 | Skeleton action recognition method based on difference guidance representation learning network |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102117116A (en) * | 2009-12-30 | 2011-07-06 | 微盟电子(昆山)有限公司 | Moving object recognition method and instruction input method based on moving object recognition |
| CN103679154A (en) * | 2013-12-26 | 2014-03-26 | 中国科学院自动化研究所 | Three-dimensional gesture action recognition method based on depth images |
| CN104750397A (en) * | 2015-04-09 | 2015-07-01 | 重庆邮电大学 | Somatosensory-based natural interaction method for virtual mine |
| CN105069413A (en) * | 2015-07-27 | 2015-11-18 | 电子科技大学 | Human body gesture identification method based on depth convolution neural network |
| CN105160310A (en) * | 2015-08-25 | 2015-12-16 | 西安电子科技大学 | 3D (three-dimensional) convolutional neural network based human body behavior recognition method |
-
2016
- 2016-07-08 CN CN201610538399.8A patent/CN106228109A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102117116A (en) * | 2009-12-30 | 2011-07-06 | 微盟电子(昆山)有限公司 | Moving object recognition method and instruction input method based on moving object recognition |
| CN103679154A (en) * | 2013-12-26 | 2014-03-26 | 中国科学院自动化研究所 | Three-dimensional gesture action recognition method based on depth images |
| CN104750397A (en) * | 2015-04-09 | 2015-07-01 | 重庆邮电大学 | Somatosensory-based natural interaction method for virtual mine |
| CN105069413A (en) * | 2015-07-27 | 2015-11-18 | 电子科技大学 | Human body gesture identification method based on depth convolution neural network |
| CN105160310A (en) * | 2015-08-25 | 2015-12-16 | 西安电子科技大学 | 3D (three-dimensional) convolutional neural network based human body behavior recognition method |
Non-Patent Citations (1)
| Title |
|---|
| ESHED OHN-BAR ET AL.: "Joint Angles Similiarities and HOG2 for Action Recognition", 《2013 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS》 * |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108052896B (en) * | 2017-12-12 | 2020-06-02 | 广东省智能制造研究所 | Human body behavior identification method based on convolutional neural network and support vector machine |
| CN108052896A (en) * | 2017-12-12 | 2018-05-18 | 广东省智能制造研究所 | Human bodys' response method based on convolutional neural networks and support vector machines |
| CN108597578B (en) * | 2018-04-27 | 2021-11-05 | 广东省智能制造研究所 | Human motion assessment method based on two-dimensional skeleton sequence |
| CN108597578A (en) * | 2018-04-27 | 2018-09-28 | 广东省智能制造研究所 | A kind of human motion appraisal procedure based on two-dimensional framework sequence |
| CN110490034A (en) * | 2018-05-14 | 2019-11-22 | 欧姆龙株式会社 | Motion analysis device, action-analysing method, recording medium and motion analysis system |
| CN108846348A (en) * | 2018-06-07 | 2018-11-20 | 四川大学 | A kind of Human bodys' response method based on three-dimensional skeleton character |
| CN109584345A (en) * | 2018-11-12 | 2019-04-05 | 大连大学 | Human motion synthetic method based on convolutional neural networks |
| CN109584345B (en) * | 2018-11-12 | 2023-10-31 | 大连大学 | Human motion synthesis method based on convolutional neural network |
| CN109670401A (en) * | 2018-11-15 | 2019-04-23 | 天津大学 | A kind of action identification method based on skeleton motion figure |
| CN110728183A (en) * | 2019-09-09 | 2020-01-24 | 天津大学 | Human body action recognition method based on attention mechanism neural network |
| CN110728183B (en) * | 2019-09-09 | 2023-09-22 | 天津大学 | Human body action recognition method of neural network based on attention mechanism |
| CN111223549A (en) * | 2019-12-30 | 2020-06-02 | 华东师范大学 | Mobile end system and method for disease prevention based on posture correction |
| CN112101262B (en) * | 2020-09-22 | 2022-09-06 | 中国科学技术大学 | Multi-feature fusion sign language recognition method and network model |
| CN112101262A (en) * | 2020-09-22 | 2020-12-18 | 中国科学技术大学 | Multi-feature fusion sign language recognition method and network model |
| CN112507940B (en) * | 2020-12-17 | 2023-08-25 | 华南理工大学 | Bone action recognition method based on differential guidance representation learning network |
| CN112507940A (en) * | 2020-12-17 | 2021-03-16 | 华南理工大学 | Skeleton action recognition method based on difference guidance representation learning network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106228109A (en) | A kind of action identification method based on skeleton motion track | |
| Song et al. | Richly activated graph convolutional network for action recognition with incomplete skeletons | |
| CN108932500B (en) | A kind of dynamic gesture identification method and system based on deep neural network | |
| Tao et al. | An object detection system based on YOLO in traffic scene | |
| CN104732208B (en) | Video human Activity recognition method based on sparse subspace clustering | |
| CN106778604B (en) | Pedestrian re-identification method based on matching convolutional neural network | |
| Lan et al. | Action recognition by hierarchical mid-level action elements | |
| CN104200237B (en) | One kind being based on the High-Speed Automatic multi-object tracking method of coring correlation filtering | |
| CN103186775B (en) | Based on the human motion identification method of mix description | |
| CN107292813A (en) | A kind of multi-pose Face generation method based on generation confrontation network | |
| CN103226835B (en) | Based on method for tracking target and the system of online initialization gradient enhancement regression tree | |
| CN105528794A (en) | Moving object detection method based on Gaussian mixture model and superpixel segmentation | |
| CN101470809B (en) | Moving object detection method based on expansion mixed gauss model | |
| CN107392131A (en) | A kind of action identification method based on skeleton nodal distance | |
| CN104021381B (en) | Human movement recognition method based on multistage characteristics | |
| Wei et al. | P3D-CTN: Pseudo-3D convolutional tube network for spatio-temporal action detection in videos | |
| CN105825233B (en) | A kind of pedestrian detection method based on on-line study random fern classifier | |
| CN105740915A (en) | Cooperation segmentation method fusing perception information | |
| CN110096961A (en) | A kind of indoor scene semanteme marking method of super-pixel rank | |
| CN104820711A (en) | Video retrieval method for figure target in complex scene | |
| CN115410119A (en) | Violent movement detection method and system based on adaptive generation of training samples | |
| Wu et al. | Single shot multibox detector for vehicles and pedestrians detection and classification | |
| CN103218829B (en) | A kind of foreground extracting method being adapted to dynamic background | |
| CN103020631B (en) | Human movement identification method based on star model | |
| CN104008374B (en) | Miner's detection method based on condition random field in a kind of mine image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20161214 |