CN109800659A - A kind of action identification method and device - Google Patents
A kind of action identification method and device Download PDFInfo
- Publication number
- CN109800659A CN109800659A CN201811604771.6A CN201811604771A CN109800659A CN 109800659 A CN109800659 A CN 109800659A CN 201811604771 A CN201811604771 A CN 201811604771A CN 109800659 A CN109800659 A CN 109800659A
- Authority
- CN
- China
- Prior art keywords
- skeleton
- character image
- skeleton character
- bone
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000009471 action Effects 0.000 title claims abstract description 26
- 238000000034 method Methods 0.000 title claims abstract description 17
- 210000000988 bone and bone Anatomy 0.000 claims abstract description 48
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 17
- 230000000875 corresponding effect Effects 0.000 claims description 20
- 239000011159 matrix material Substances 0.000 claims description 16
- 238000010606 normalization Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 6
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000000295 complement effect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012731 temporal analysis Methods 0.000 description 1
- 238000000700 time series analysis Methods 0.000 description 1
Abstract
The invention proposes a kind of action identification method and devices, comprising: obtains the skeleton data of object to be identified during exercise in video；The bone sequence of the object to be identified is generated according to the skeleton data；Skeleton character image corresponding with the bone sequence is generated, includes several skeleton points in the skeleton character image；Skeleton character image is input to default convolutional neural networks model to classify, obtains action classification corresponding with the skeleton character figure.The problem of the problem of action recognition is converted to the classification of bone sequence image by the present invention, is converted to skeleton character image for bone sequence, then again to skeleton character image classification, so that identification is more accurate, it is more efficient.
Description
Technical field
The present invention relates to identification fields, and in particular to a kind of action identification method and device.
Background technique
Human action's identification has multiple modalities, such as appearance, depth, light stream and body bone etc..Work as in these mode
In, dynamic human skeletal can usually complement each other with other mode, convey important information.Therefore, can by bone sequence come
Carry out human action identification.
However the method that existing bone carries out action recognition is the coordinate of skeleton point directly to be connected for an one-dimensional length
Vector, and Time-Series analysis is carried out to it, the accuracy rate of this recognition methods is low.
Therefore the present invention provides a kind of action identification method and device to solve the deficiencies in the prior art.
Summary of the invention
The present invention is intended to provide a kind of action identification method and device, solve the action classification identification of current bone sequence not
Accurate problem.
According to an aspect of the present invention, a kind of action identification method is provided, comprising:
Obtain the skeleton data of object to be identified during exercise in video；
The bone sequence of the object to be identified is generated according to the skeleton data；
Skeleton character image corresponding with the bone sequence is generated, includes several bones in the skeleton character image
Point；
Skeleton character image is input to default convolutional neural networks model to classify, is obtained and the skeleton character figure
Corresponding action classification.
Further, skeleton character image corresponding with the bone sequence is generated, comprising:
The three-dimensional point coordinate of bone sequence in frame image every in video is arranged as Three-channel data according to preset order；
The Three-channel data is sequentially arranged as triple channel matrix；
The triple channel matrix is normalized, skeleton character image is obtained.
Further, the triple channel matrix is normalized, obtains skeleton character image, comprising:
The normalization is shown below:
Wherein,For the pixel value at position coordinates (i, j) on c-th of channel of skeleton character image；WithPoint
Not Wei on c-th of channel of skeleton character image pixel minimum value and maximum value；Round () is bracket function.
Further, skeleton character image default convolutional neural networks model is input to classify, obtain with it is described
The corresponding action classification of skeleton character figure, comprising:
Utilize the feature of skeleton character image described in default convolutional neural networks model extraction；
Using full articulamentum by the Feature Conversion be feature vector；
Determine that the type of the skeleton character image, the type are the object to be identified according to described eigenvector
Action classification.
According to a further aspect of the invention, it discloses a kind of action recognition devices, comprising:
Module is obtained, for obtaining the skeleton data of object to be identified during exercise in video；
Generation module, for generating the bone sequence of the object to be identified according to the skeleton data；
Characteristic image module, for generating skeleton character image corresponding with the bone sequence, the skeleton character figure
It include several skeleton points as in；
Determining module is classified for skeleton character image to be input to default convolutional neural networks model, obtain with
The corresponding action classification of the skeleton character figure.
Further, the characteristic image module includes:
First sorting sub-module, for by the three-dimensional point coordinate of bone sequence in frame image every in video according to preset order
It is arranged as Three-channel data；
Second sorting sub-module, for being sequentially arranged the Three-channel data for triple channel matrix；
It normalizes submodule and obtains skeleton character image for the triple channel matrix to be normalized.
Further, the normalization submodule, is used for,
The normalization is shown below:
Wherein,For the pixel value at position coordinates (i, j) on c-th of channel of skeleton character image；WithPoint
Not Wei on c-th of channel of skeleton character image pixel minimum value and maximum value；Round () is bracket function.
Further, the determining module includes:
Extracting sub-module, for the feature using skeleton character image described in default convolutional neural networks model extraction；
Transform subblock is used to using full articulamentum be feature vector by the Feature Conversion；
Submodule is determined, for determining that the type of the skeleton character image, the type be according to described eigenvector
The action classification of the object to be identified.
The beneficial effect of the technical program is compared with the immediate prior art:
Technical solution provided by the invention obtains the skeleton data of object to be identified during exercise in video, then by bone
Bone data are converted to the bone sequence of object to be identified, then bone sequence is converted to skeleton character image, are set using preset
Switching network is ranked up all skeleton points in skeleton character image, finally using convolutional neural networks to the bone after sequence
Characteristic image is classified, and motion characteristic corresponding with the skeleton character figure is obtained.The present invention will turn the problem of action recognition
The problem of being melted into the classification of bone sequence image, is converted to skeleton character image for bone sequence, then again to skeleton character image
Classification, so that identification is more accurate, it is more efficient.
Detailed description of the invention
Fig. 1 is flow chart of the method for the present invention.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiments of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill people
Member's every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
As shown in Figure 1, process is as follows the present invention provides a kind of action identification method:
S101, the skeleton data of object to be identified during exercise in video is obtained；
S102, the bone sequence that the object to be identified is generated according to the skeleton data；
S103, skeleton character image corresponding with the bone sequence is generated, includes several in the skeleton character image
A skeleton point；
S104, skeleton character image is input to default convolutional neural networks model classifies, obtain and the bone
The corresponding action classification of characteristic pattern.
In the embodiment of the present application, the skeleton data of object to be identified during exercise in video is obtained, then by bone
Data are converted to the bone sequence of object to be identified, then bone sequence is converted to skeleton character image, finally utilize convolution mind
Classify through network to skeleton character image, obtains motion characteristic corresponding with the skeleton character figure.The present invention will act
The problem of identification, is converted to the problem of bone sequence image classification, and bone sequence is converted to skeleton character image, then right again
Skeleton character image classification, so that identification is more accurate, it is more efficient.
In some embodiments of the present application, the bone sequence v of a T frame, in t frame, the seat of k-th of skeleton point are given
Mark is expressed asWherein, t ∈ [1,2 ..., T], k ∈ [1,2 ..., N], N represent the number of skeleton point in a frame
Mesh.The skeleton data of t frame is expressed as St={ J1,J2,…,JN}.Generate skeleton character image corresponding with the bone sequence
Mainly include three steps:
Step 1: the three-dimensional point coordinate of bone sequence in frame image every in video is arranged as triple channel according to preset order
Data；
Step 2: being sequentially arranged the Three-channel data for triple channel matrix；
Step 3: the triple channel matrix is normalized, skeleton character image is obtained.
In step 1, regard the coordinate (x, y, z) of three dimensions as three channels.By taking x dimension as an example, by StIn x
Dimension value according to a predefined sequence O=(o1,o2,…,ok,…,oK) it is arranged as a feature vector, obtain spy
Levy vector ftX channel characteristics vector ft x, whereinThe O that puts in order is determined
Skeleton point closes on relationship in the picture.In step 2, by the triple channel skeleton character of all framesAccording to time elder generation
It is sequentially arranged as the matrix M an of triple channel afterwards.By taking the channel x as an example,The size of M be 3 × T ×
K, wherein T is the length of video sequence, and K is the length of O of putting in order.
In step 3, M is normalized as the following formula, is quantified as a RGB image I:
Wherein,For the pixel value at position coordinates (i, j) on c-th of the channel image I；WithRespectively bone
The minimum value and maximum value of pixel on c-th of channel of characteristic image；Round () is bracket function.By to the element in matrix
The minimum value in channel is subtracted, then is normalized divided by value variable quantity maximum in all channels.Then value is quantified
To the section of [0,255].
In some embodiments of the present application, skeleton character image is input to default convolutional neural networks model and is divided
Class obtains action classification corresponding with the skeleton character figure, comprising:
Utilize the feature of skeleton character image described in default convolutional neural networks model extraction；
Using full articulamentum by the Feature Conversion be feature vector；
Determine that the type of the skeleton character image, the type are the object to be identified according to described eigenvector
Action classification.
Based on identical inventive concept, the present invention also provides a kind of action recognition devices, comprising:
Module is obtained, for obtaining the skeleton data of object to be identified during exercise in video；
Generation module, for generating the bone sequence of the object to be identified according to the skeleton data；
Characteristic image module, for generating skeleton character image corresponding with the bone sequence, the skeleton character figure
It include several skeleton points as in；
Determining module is classified for skeleton character image to be input to default convolutional neural networks model, obtain with
The corresponding action classification of the skeleton character figure.
Optionally, the characteristic image module includes:
First sorting sub-module, for by the three-dimensional point coordinate of bone sequence in frame image every in video according to preset order
It is arranged as Three-channel data；
Second sorting sub-module, for being sequentially arranged the Three-channel data for triple channel matrix；
It normalizes submodule and obtains skeleton character image for the triple channel matrix to be normalized.
Optionally, the normalization submodule, is used for,
The normalization is shown below:
Wherein,For the pixel value at position coordinates (i, j) on c-th of channel of skeleton character image；WithPoint
Not Wei on c-th of channel of skeleton character image pixel minimum value and maximum value；Round () is bracket function.
Optionally, the determining module includes:
Extracting sub-module, for the feature using skeleton character image described in default convolutional neural networks model extraction；
Transform subblock is used to using full articulamentum be feature vector by the Feature Conversion；
Submodule is determined, for determining that the type of the skeleton character image, the type be according to described eigenvector
The motion characteristic of the object to be identified.
Particular embodiments described above has carried out further in detail the purpose of the present invention, technical scheme and beneficial effects
Describe in detail bright, it should be understood that the above is only a specific embodiment of the present invention, is not intended to restrict the invention, it is all
Within the spirit and principles in the present invention, any modification, equivalent replacement, improvement and so on should be included in protection of the invention
Within the scope of.
Claims (8)
1. a kind of action identification method characterized by comprising
Obtain the skeleton data of object to be identified during exercise in video；
The bone sequence of the object to be identified is generated according to the skeleton data；
Skeleton character image corresponding with the bone sequence is generated, includes several skeleton points in the skeleton character image；
Skeleton character image is input to default convolutional neural networks model to classify, is obtained corresponding with the skeleton character figure
Action classification.
2. the method according to claim 1, wherein generating skeleton character figure corresponding with the bone sequence
Picture, comprising:
The three-dimensional point coordinate of bone sequence in frame image every in video is arranged as Three-channel data according to preset order；
The Three-channel data is sequentially arranged as triple channel matrix；
The triple channel matrix is normalized, skeleton character image is obtained.
3. according to the method described in claim 2, obtaining it is characterized in that, the triple channel matrix is normalized
Skeleton character image, comprising:
The normalization is shown below:
Wherein,For the pixel value at position coordinates (i, j) on c-th of channel of skeleton character image；WithRespectively bone
The minimum value and maximum value of pixel on c-th of channel of bone characteristic image；Round () is bracket function.
4. the method according to claim 1, wherein skeleton character image is input to default convolutional neural networks
Model is classified, and action classification corresponding with the skeleton character figure is obtained, comprising:
Utilize the feature of skeleton character image described in default convolutional neural networks model extraction；
Using full articulamentum by the Feature Conversion be feature vector；
Determine that the type of the skeleton character image, the type are the movement of the object to be identified according to described eigenvector
Classification.
5. a kind of action recognition device characterized by comprising
Module is obtained, for obtaining the skeleton data of object to be identified during exercise in video；
Generation module, for generating the bone sequence of the object to be identified according to the skeleton data；
Characteristic image module, for generating corresponding with bone sequence skeleton character image, in the skeleton character image
Including several skeleton points；
Determining module is classified for skeleton character image to be input to default convolutional neural networks model, obtain with it is described
The corresponding action classification of skeleton character figure.
6. device according to claim 5, which is characterized in that the characteristic image module includes:
First sorting sub-module, for arranging the three-dimensional point coordinate of bone sequence in frame image every in video according to preset order
For Three-channel data；
Second sorting sub-module, for being sequentially arranged the Three-channel data for triple channel matrix；
It normalizes submodule and obtains skeleton character image for the triple channel matrix to be normalized.
7. device according to claim 6, which is characterized in that the normalization submodule is used for,
The normalization is shown below:
Wherein,For the pixel value at position coordinates (i, j) on c-th of channel of skeleton character image；WithRespectively bone
The minimum value and maximum value of pixel on c-th of channel of bone characteristic image；Round () is bracket function.
8. device according to claim 5, which is characterized in that the determining module includes:
Extracting sub-module, for the feature using skeleton character image described in default convolutional neural networks model extraction；
Transform subblock is used to using full articulamentum be feature vector by the Feature Conversion；
Submodule is determined, for determining that the type of the skeleton character image, the type are described according to described eigenvector
The action classification of object to be identified.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811604771.6A CN109800659B (en) | 2018-12-26 | 2018-12-26 | Action recognition method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811604771.6A CN109800659B (en) | 2018-12-26 | 2018-12-26 | Action recognition method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109800659A true CN109800659A (en) | 2019-05-24 |
| CN109800659B CN109800659B (en) | 2021-05-25 |
Family
ID=66557740
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811604771.6A Active CN109800659B (en) | 2018-12-26 | 2018-12-26 | Action recognition method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109800659B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111476115A (en) * | 2020-03-23 | 2020-07-31 | 深圳市联合视觉创新科技有限公司 | Human behavior recognition method, device and equipment |
| CN112861808A (en) * | 2021-03-19 | 2021-05-28 | 泰康保险集团股份有限公司 | Dynamic gesture recognition method and device, computer equipment and readable storage medium |
| CN113229832A (en) * | 2021-03-24 | 2021-08-10 | 清华大学 | System and method for acquiring human motion information |
| CN113537121A (en) * | 2021-07-28 | 2021-10-22 | 浙江大华技术股份有限公司 | Identity recognition method and device, storage medium and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104318248A (en) * | 2014-10-21 | 2015-01-28 | 北京智谷睿拓技术服务有限公司 | Action recognition method and action recognition device |
| CN106203503A (en) * | 2016-07-08 | 2016-12-07 | 天津大学 | A kind of action identification method based on skeleton sequence |
| CN106203363A (en) * | 2016-07-15 | 2016-12-07 | 中国科学院自动化研究所 | Human skeleton motion sequence Activity recognition method |
| US20170161555A1 (en) * | 2015-12-04 | 2017-06-08 | Pilot Ai Labs, Inc. | System and method for improved virtual reality user interaction utilizing deep-learning |
-
2018
- 2018-12-26 CN CN201811604771.6A patent/CN109800659B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104318248A (en) * | 2014-10-21 | 2015-01-28 | 北京智谷睿拓技术服务有限公司 | Action recognition method and action recognition device |
| US20170161555A1 (en) * | 2015-12-04 | 2017-06-08 | Pilot Ai Labs, Inc. | System and method for improved virtual reality user interaction utilizing deep-learning |
| CN106203503A (en) * | 2016-07-08 | 2016-12-07 | 天津大学 | A kind of action identification method based on skeleton sequence |
| CN106203363A (en) * | 2016-07-15 | 2016-12-07 | 中国科学院自动化研究所 | Human skeleton motion sequence Activity recognition method |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111476115A (en) * | 2020-03-23 | 2020-07-31 | 深圳市联合视觉创新科技有限公司 | Human behavior recognition method, device and equipment |
| CN111476115B (en) * | 2020-03-23 | 2023-08-29 | 深圳市联合视觉创新科技有限公司 | Human behavior recognition method, device and equipment |
| CN112861808A (en) * | 2021-03-19 | 2021-05-28 | 泰康保险集团股份有限公司 | Dynamic gesture recognition method and device, computer equipment and readable storage medium |
| CN112861808B (en) * | 2021-03-19 | 2024-01-23 | 泰康保险集团股份有限公司 | Dynamic gesture recognition method, device, computer equipment and readable storage medium |
| CN113229832A (en) * | 2021-03-24 | 2021-08-10 | 清华大学 | System and method for acquiring human motion information |
| CN113537121A (en) * | 2021-07-28 | 2021-10-22 | 浙江大华技术股份有限公司 | Identity recognition method and device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109800659B (en) | 2021-05-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109359538B (en) | Training method of convolutional neural network, gesture recognition method, device and equipment | |
| CN109800659A (en) | A kind of action identification method and device | |
| CN110598610B (en) | Target significance detection method based on neural selection attention | |
| CN107256246B (en) | printed fabric image retrieval method based on convolutional neural network | |
| CN110175595A (en) | Human body attribute recognition approach, identification model training method and device | |
| CN106874826A (en) | Face key point-tracking method and device | |
| JP7286010B2 (en) | Human body attribute recognition method, device, electronic device and computer program | |
| CN108121957A (en) | The method for pushing and device of U.S. face material | |
| CN109409994A (en) | The methods, devices and systems of analog subscriber garments worn ornaments | |
| CN109145837A (en) | Face emotion identification method, device, terminal device and storage medium | |
| CN105005980B (en) | Image processing method and device | |
| CN108388882A (en) | Based on the gesture identification method that the overall situation-part is multi-modal RGB-D | |
| CN107958218A (en) | A kind of real-time gesture knows method for distinguishing | |
| CN103140862A (en) | User interface system and method of operation thereof | |
| CN102567716A (en) | Face synthetic system and implementation method | |
| CN107018330A (en) | A kind of guidance method and device of taking pictures in real time | |
| CN105022999A (en) | Man code company real-time acquisition system | |
| CN107172354A (en) | Method for processing video frequency, device, electronic equipment and storage medium | |
| CN103065282A (en) | Image fusion method based on sparse linear system | |
| CN108694719A (en) | image output method and device | |
| CN110097616B (en) | Combined drawing method and device, terminal equipment and readable storage medium | |
| CN110163567A (en) | Classroom roll calling system based on multitask concatenated convolutional neural network | |
| CN110322398A (en) | Image processing method, device, electronic equipment and computer readable storage medium | |
| CN110472600A (en) | The identification of eyeground figure and its training method, device, equipment and storage medium | |
| CN113747640A (en) | Intelligent central control method and system for digital exhibition hall lamplight |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 211135 floor 3, building 3, Qilin artificial intelligence Industrial Park, 266 Chuangyan Road, Nanjing, Jiangsu Patentee after: Zhongke Nanjing artificial intelligence Innovation Research Institute Patentee after: INSTITUTE OF AUTOMATION, CHINESE ACADEMY OF SCIENCES Address before: 211135 3rd floor, building 3, 266 Chuangyan Road, Jiangning District, Nanjing City, Jiangsu Province Patentee before: NANJING ARTIFICIAL INTELLIGENCE CHIP INNOVATION INSTITUTE, INSTITUTE OF AUTOMATION, CHINESE ACADEMY OF SCIENCES Patentee before: INSTITUTE OF AUTOMATION, CHINESE ACADEMY OF SCIENCES |
|
| CP03 | Change of name, title or address |