CN106228121A - Gesture feature recognition methods and device - Google Patents
Gesture feature recognition methods and device Download PDFInfo
- Publication number
- CN106228121A CN106228121A CN201610559968.7A CN201610559968A CN106228121A CN 106228121 A CN106228121 A CN 106228121A CN 201610559968 A CN201610559968 A CN 201610559968A CN 106228121 A CN106228121 A CN 106228121A
- Authority
- CN
- China
- Prior art keywords
- gesture
- frame
- depth map
- subsequence
- sequence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of gesture feature recognition methods and device.This gesture feature recognition methods includes: calculates the similarity between frame and the frame of gesture depth map sequence, obtains result of calculation；According to result of calculation, gesture depth map sequence is resolved into multiple subsequence；Extracting the key node of each subsequence in multiple subsequence, wherein, key node is to meet pre-conditioned frame in each subsequence；Key node is formed the set of keypoints of gesture depth map sequence；And carry out gesture identification according to set of keypoints, obtain gesture identification result.By the present invention, solve temporal redundancy information and cause the descriptive low problem of gesture feature more.
Description
Technical field
The present invention relates to gesture identification field, in particular to a kind of gesture feature recognition methods and device.
Background technology
Dynamic gesture identification method based on depth map sequence mainly utilizes the global characteristics of depth map sequence to carry out gesture
Classification and Identification, at present relatively the most classical algorithm mainly has the action recognition algorithm mapped based on Depth Motion and based on four-dimensional method
The action recognition algorithm of histogram vector.
Action recognition algorithm based on depth map Motion mapping: depth map sequence is regarded as an entirety, to each frame figure
As carrying out the mapping in three directions, respectively obtain front view, top view, a left side attempt.Ask the most respectively in every width mapping graph time domain
By difference, difference between consecutive frame, judges whether this pixel moves.Motion conditions is added up and has just obtained three
Map the Motion mapping figure on direction.Ask the histogram of gradients of every width mapping graph, and the gradient will tried to achieve in three width figures further
Set of histograms altogether, constitutes the global characteristics of gesture sequence.Carry out the training of grader by this feature, obtain gesture identification
Result.
Based on four-dimensional normal vector histogrammic action recognition algorithm: depth map sequence to be regarded as the song of a space-time
Face.Point on this curved surface of each pixel.Each pixel is asked time-space domain gradient, and according to space-time surface equation,
Time-space domain gradient is converted to the normal vector of curved surface.The mode of following mapping adds up four-dimensional normal vector to obtain four-dimensional normal direction
Amount rectangular histogram, the mode of mapping is to utilize 120 apex coordinates composition mapping matrixes of positive 600 cell spaces, by four-dimension normal vector to
Project on these 120 directions, thus obtain the rectangular histogram of one 120 dimension.Additionally, depth map sequence has also been carried out point by the method
Block processes, and calculates four-dimensional normal vector rectangular histogram, and the constitutive characteristic vector that all of rectangular histogram is stitched together in each piecemeal.
Train grader by characteristic vector further, carry out action recognition.
Existing Gesture Recognition Algorithm based on depth map sequence, owing to the most fully analyzing the feature of gesture motion, leads
Global characteristics two problems of existence that cause is extracted: first, due to closely similar between gesture depth map sequence frame and frame, cause
The when of carrying out global characteristics extraction, there is substantial amounts of temporal redundancy information in feature, this temporal redundancy information makes gesture feature
Descriptive reduction.Second, current depth map gesture feature extraction algorithm does not accounts for same gesture due to the difference of speed
Cause feature to have the situation of the biggest difference, cause the robustness of Gesture Recognition Algorithm to reduce.
Cause the descriptive low problem of gesture feature more for temporal redundancy information in correlation technique, the most not yet propose
Effective solution.
Summary of the invention
Present invention is primarily targeted at a kind of gesture feature recognition methods of offer and device, to solve temporal redundancy information
Cause the descriptive low problem of gesture feature more.
To achieve these goals, according to an aspect of the invention, it is provided a kind of gesture feature recognition methods, the party
Method includes: calculates the similarity between frame and the frame of gesture depth map sequence, obtains result of calculation；According to result of calculation by gesture
Depth map sequence resolves into multiple subsequence；Extract the key node of each subsequence in multiple subsequence, wherein, key node
For each subsequence meets pre-conditioned frame；Key node is formed the set of keypoints of gesture depth map sequence；And
Carry out gesture identification according to set of keypoints, obtain gesture identification result.
Further, the similarity calculated between frame and the frame of gesture depth map sequence includes: the figure to depth map sequence
As carrying out piecemeal process, obtain multiple image block；Calculate the gradient direction angle of all pixels in each image block, by each
In image block, the maximum of gradient direction angle is as the orientation angle of each image block；With the orientation angle of all image blocks it is
Elementary composition characteristic vector；The similarity between frame and the frame of gesture depth map sequence is calculated according to characteristic vector.
Further, the similarity calculated according to characteristic vector between frame and the frame of gesture depth map sequence includes: pass through
Gauss similar function calculates the similar matrix of characteristic vector composition；It is weighted similar matrix processing, the phase after being weighted
Like matrix；The similarity between frame and the frame of gesture depth map sequence is calculated according to the similar matrix after weighting.
Further, according to result of calculation, gesture depth map sequence is resolved into multiple subsequence to include: according to vertex set
Setting up non-directed graph model with adjacency matrix, wherein, vertex set is the set of all depth maps in depth map sequence, in adjacency matrix
Any two two field pictures of element representation between similarity, adjacency matrix is made up of the similar matrix after weighting；Graph model is entered
Row divides, and wherein, graph model is carried out division and includes: the similarity matrix after weighting is converted to Laplacian Matrix, solves
Laplacian Matrix, obtains eigenvalue and characteristic vector；In characteristic vector second little characteristic vector is as optimal vector；To
Point in excellent vector clusters.
Further, according to result of calculation, gesture depth map sequence is resolved into multiple subsequence also to include: to current deep
Degree graphic sequence carries out two points of iteration clusters, wherein, current depth graphic sequence carries out two points of iteration clusters and includes: by current sequence
Optimal vector to gather be two classes, obtain two class optimal vectors, according to the similarity average of current sequence and two class optimal vectors
The relation of similarity average determines the condition of iteration stopping.
Further, extract the key node of each subsequence in multiple subsequence to include: calculate in each subsequence
Euclidean distance between any two frames, obtains multiple Euclidean distance；Using frame minimum for Euclidean distance in each subsequence as son
The key node of sequence.
To achieve these goals, according to a further aspect in the invention, a kind of gesture feature identification device is additionally provided, should
Device includes: computing unit, the similarity between frame and the frame calculating gesture depth map sequence, obtains result of calculation；Point
Solve unit, for gesture depth map sequence being resolved into multiple subsequence according to result of calculation；Extraction unit, is used for extracting multiple
The key node of each subsequence in subsequence, wherein, key node is to meet pre-conditioned frame in each subsequence；Combination
Unit, for forming the set of keypoints of gesture depth map sequence by key node；And recognition unit, for according to key point
Set carries out gesture identification, obtains gesture identification result.
Further, computing unit includes: decomposing module, for the image of depth map sequence is carried out piecemeal process,
To multiple image blocks；First computing module, for calculating the gradient direction angle of all pixels in each image block, by each
In image block, the maximum of gradient direction angle is as the orientation angle of each image block；Composite module, for all images
The orientation angle of block is elementary composition characteristic vector；Second computing module, for calculating gesture depth map sequence according to characteristic vector
Similarity between frame and the frame of row.
Further, the second computing module includes: the first calculating sub module, for calculating feature by Gauss similar function
The similar matrix of vector composition；Weighting submodule, for being weighted process, the similar square after being weighted to similar matrix
Battle array；Second calculating sub module, the phase between frame and the frame calculating gesture depth map sequence according to the similar matrix after weighting
Like property.
Further, resolving cell includes: set up module, for setting up undirected artwork according to vertex set and adjacency matrix
Type, wherein, vertex set is the set of all depth maps in depth map sequence, any two two field pictures of element representation in adjacency matrix
Between similarity, adjacency matrix is made up of the similar matrix after weighting；Divide module, for graph model is divided, its
In, graph model is carried out division and includes: the similarity matrix after weighting is converted to Laplacian Matrix, solves Laplce's square
Battle array, obtains eigenvalue and characteristic vector；In characteristic vector second little characteristic vector is as optimal vector；To in optimal vector
Point clusters.
The present invention, by the similarity between frame and the frame of calculating gesture depth map sequence, obtains result of calculation；According to meter
Calculate result and gesture depth map sequence is resolved into multiple subsequence；Extract the key node of each subsequence in multiple subsequence,
Wherein, key node meets pre-conditioned frame in being each subsequence；Key node is formed the pass of gesture depth map sequence
Key point set；And carry out gesture identification according to set of keypoints, obtain gesture identification result, solve temporal redundancy information many
Cause the descriptive low problem of gesture feature, and then reach to promote the illustrative effect of gesture feature.
Accompanying drawing explanation
The accompanying drawing of the part constituting the application is used for providing a further understanding of the present invention, and the present invention's is schematic real
Execute example and illustrate for explaining the present invention, being not intended that inappropriate limitation of the present invention.In the accompanying drawings:
Fig. 1 is the flow chart of gesture feature recognition methods according to embodiments of the present invention；And
Fig. 2 is the schematic diagram of gesture feature identification device according to embodiments of the present invention.
Detailed description of the invention
It should be noted that in the case of not conflicting, the embodiment in the application and the feature in embodiment can phases
Combination mutually.Describe the present invention below with reference to the accompanying drawings and in conjunction with the embodiments in detail.
In order to make those skilled in the art be more fully understood that the application scheme, below in conjunction with in the embodiment of the present application
Accompanying drawing, is clearly and completely described the technical scheme in the embodiment of the present application, it is clear that described embodiment is only
The embodiment of the application part rather than whole embodiments.Based on the embodiment in the application, ordinary skill people
The every other embodiment that member is obtained under not making creative work premise, all should belong to the model of the application protection
Enclose.
It should be noted that term " first " in the description and claims of this application and above-mentioned accompanying drawing, "
Two " it is etc. for distinguishing similar object, without being used for describing specific order or precedence.Should be appreciated that so use
Data can exchange in the appropriate case, in order to embodiments herein described herein.Additionally, term " includes " and " tool
Have " and their any deformation, it is intended that cover non-exclusive comprising, such as, contain series of steps or unit
Process, method, system, product or equipment are not necessarily limited to those steps or the unit clearly listed, but can include the most clear
That list to Chu or for intrinsic other step of these processes, method, product or equipment or unit.
Embodiments provide a kind of gesture feature recognition methods.
Fig. 1 is the flow chart of gesture feature recognition methods according to embodiments of the present invention, as it is shown in figure 1, the method includes
Following steps:
Step S101: calculate the similarity between frame and the frame of gesture depth map sequence, obtain result of calculation.
Gesture depth map sequence includes multiframe, due to closely similar, opponent between frame and the frame of gesture depth map sequence
The when that gesture depth map carrying out global characteristics extraction, there is substantial amounts of sequential redundancy, therefore, first calculate gesture depth map
Similarity between frame and the frame of sequence, can simplify calculating process during follow-up.Calculate gesture depth map sequence
Similarity between frame and frame can be calculated by multiple computational methods, alternatively, calculates the frame of gesture depth map sequence
And the similarity between frame can be by the following method: the image of depth map sequence is carried out piecemeal process, obtains multiple image
Block；Calculate the gradient direction angle of all pixels in each image block, by the maximum of gradient direction angle in each image block
It is worth the orientation angle as each image block；It is elementary composition characteristic vector with the orientation angle of all image blocks；According to feature
Vector calculates the similarity between frame and the frame of gesture depth map sequence.
Alternatively, the similarity calculated according to characteristic vector between frame and the frame of gesture depth map sequence includes: by height
This similar function calculates the similar matrix of characteristic vector composition；It is weighted similar matrix processing, similar after being weighted
Matrix；The similarity between frame and the frame of gesture depth map sequence is calculated according to the similar matrix after weighting.
Step S102: gesture depth map sequence is resolved into multiple subsequence according to result of calculation.
Alternatively, according to result of calculation, gesture depth map sequence is resolved into multiple subsequence to include: according to vertex set and
Adjacency matrix sets up non-directed graph model, and wherein, vertex set is the set of all depth maps in depth map sequence, in adjacency matrix
Similarity between any two two field pictures of element representation, adjacency matrix is made up of the similar matrix after weighting；Graph model is carried out
Divide, wherein, graph model is carried out division and includes: the similarity matrix after weighting is converted to Laplacian Matrix, solves and draw
This matrix of pula, obtains eigenvalue and characteristic vector；In characteristic vector second little characteristic vector is as optimal vector；To optimum
Point in vector clusters.
Alternatively, according to result of calculation, gesture depth map sequence is resolved into multiple subsequence also to include: to current depth
Graphic sequence carries out two points of iteration clusters, wherein, current depth graphic sequence carries out two points of iteration clusters and includes: by current sequence
It is two classes that optimal vector is gathered, and obtains two class optimal vectors, according to similarity average and the phase of two class optimal vectors of current sequence
Relation like property average determines the condition of iteration stopping.
Gesture sequence can be decomposed by this embodiment according to similar matrix, and the result of decomposition is that similar frame is drawn
Assigning to same class, dissimilar matrix is divided into different classes.Gesture sequence is decomposed into multiple subprocess.Be conducive to overcoming speed
Diversity between the gesture that degree change difference is brought.
Step S103: extract the key node of each subsequence in multiple subsequence.
Key node is to meet pre-conditioned frame in each subsequence, and key node can be optimum in each subsequence
Representational frame, alternatively, extracts the key node of each subsequence in multiple subsequence and can pass through following steps: each
Subsequence calculates the Euclidean distance between any two frames, obtains multiple Euclidean distance；By Euclidean distance in each subsequence
Little frame is as the key node of subsequence.
Step S104: key node is formed the set of keypoints of gesture depth map sequence.
In being extracted multiple subsequence after the key node of each subsequence, by the key node group of all subsequences
Become the set of keypoints of gesture depth map sequence.Owing to key node is optimum representational frame in each subsequence, therefore close
Key point set eliminates sequential redundancy, therefore, it is possible to overcome the descriptive low problem of gesture feature.
Step S105: carry out gesture identification according to set of keypoints, obtains gesture identification result.
After set of keypoints, the set of keypoints of key node composition is carried out gesture identification, obtains gesture identification
Result.The set of keypoints forming key node is carried out gesture identification and can be identified by multiple method, is not limited to certain
A kind of concrete recognition methods.
In prior art, when global characteristics extracts there is substantial amounts of sequential redundancy in feature, and sequential redundancy makes
Gesture feature descriptive low, the embodiment of the present invention is by extracting the key node in multiple subsequences, when effectively reducing
Sequence redundancy, solves the descriptive low problem of gesture feature, and meanwhile, the depth map gesture feature of prior art extracts to be calculated
Method does not accounts for same gesture and causes feature to have the situation of the biggest difference due to the difference of speed, causes the Shandong of Gesture Recognition Algorithm
Rod is low, the scheme of the embodiment of the present invention gesture sequence decomposition and extract key node time consider same gesture due to speed
The feature difference that the difference of degree causes, improves the robustness of Gesture Recognition Algorithm.
This embodiment uses the similarity between the frame and the frame that calculate gesture depth map sequence, obtains result of calculation；According to
Gesture depth map sequence is resolved into multiple subsequence by result of calculation；Extract the crucial joint of each subsequence in multiple subsequence
Point, wherein, key node is to meet pre-conditioned frame in each subsequence；Key node is formed gesture depth map sequence
Set of keypoints；And carry out gesture identification according to set of keypoints, obtain gesture identification result, solve temporal redundancy information
Cause the descriptive low problem of gesture feature more, and then reach to promote the illustrative effect of gesture feature.
Below in conjunction with a specific embodiment, the gesture feature recognition methods of the present invention is further illustrated:
The first step, calculates the similarity between frame and the frame of gesture depth map sequence, obtains result of calculation.
The embodiment of the present invention is for the similarity between quantized frame and frame, it is proposed that gesture feature based on direction extracts to be calculated
Method, specifically includes following steps:
For given depth map I, Sobel operator is utilized to seek transverse gradients G of each pixel respectivelyxAnd Gy, such as formula
(1) shown in:
Wherein,It it is the template of Sobel algorithm.
Seek the orientation angle of gradient further, as shown in formula (2):
The trend at depth map edge can be portrayed in direction due to gradient, but the edge of depth map is the sharpest keen, this
Bright embodiment carries out piecemeal process to image further, divides an image into the block of k*k size.Statistics ladder in each piecemeal
Degree orientation angle, takes the deflection angle as current block of maximum.The feature of ultimate depth figure can be expressed as with all points
The deflection of block is characteristic vector F={d of element1,d2,...,dn, this feature vector be namely based on the gesture feature in direction to
Amount, it can describe the similarity between frame and frame, wherein, d1To dnRepresent the deflection of first piece to n-th piece respectively.
Similarity similarity matrix between frame and frame represents, wherein each element representation any two of similar matrix
Characteristic similarity between frame depth map, is tried to achieve by Gauss similar function, as shown in formula (3):
But depth map sequence is sequence orderly in time domain, in time domain to be reduced, the image of wide apart is similar
Property, similar matrix is weighted processing by the present invention further, and weights are tried to achieve by formula (4):
Final similarity between frame and frame is expressed as wi,j=si,j*sti,j。
Second step, resolves into multiple subsequence according to result of calculation by gesture depth map sequence.
For solving the resolution problem of gesture sequence, the embodiment of the present invention proposes gesture decomposition method based on spectral clustering,
Specifically include following steps:
First non-directed graph model G=(V, E), wherein V={v are built1,v2,...,vnRepresent vertex set, depth map sequence
In each width depth map represent a summit.E={s11,s12,...,sij,...,snnRepresent adjacency matrix, the most each unit
Element represents the similarity between any two two field pictures.Therefore adjacency matrix is made up of above-mentioned similarity matrix.Similarity measurement is big
Representing that two summits are connected in 0, otherwise summit is not attached to.
In order to depth map sequence is decomposed, need above-mentioned graph model is divided.And spectral clustering can be by
Graph partition problem is converted into the Solve problems of matrix, method particularly includes: similarity matrix is converted to Laplacian MatrixWherein D represents the degree matrix of figure, as shown in formula (5):
Solve Laplacian Matrix, obtain eigenvalue and characteristic vector.Wherein the second little characteristic vector is referred to as Fiedler
Vector (namely optimal vector), is approaching of potential function, and it illustrates the optimal solution that figure divides, and this point has been passed through real
Verify bright, therefore choose Fiedler vector and the point in this vector is clustered.
Class number uncertain problem during in order to solve to cluster, the embodiment of the present invention proposes two points of iteration clusters and calculates
Method.Method particularly includes: current depth sequence is constantly carried out to two points of iteration cluster, will the Fiedler vector of current sequence
Gathering is two classes, and the embodiment of the present invention is according to the similarity average of current sequence, and the pass of the similarity average of two subclass
System arranges the condition of iteration stopping.Assuming that current sequence is A, the similarity average between frame and frame is dA, two subsequence B
It is respectively d with the similarity average of CB, dC, then the condition of iteration stopping is calculated by formula (6).
3rd step, extracts the key node of each subsequence in multiple subsequence.
For solving to extract the problem of key node from the gesture sequence decomposed, the embodiment of the present invention is in each subsequence
Calculate the Euclidean distance between any two frames, choose the minimum frame of between other all frames Euclidean distance as current subsequence
Key node.Choose shown in mode such as formula (7):
The set of keypoints of whole gesture sequence is expressed as:
4th step, forms the set of keypoints of gesture depth map sequence by key node.
5th step, carries out gesture feature extraction and dynamic hand gesture recognition based on this set of keypoints, obtains gesture identification knot
Really.
The embodiment of the present invention proposes the dynamic hand gesture recognition algorithm decomposed based on gesture.By gesture depth map sequence according to
Similarity between frame and frame is decomposed into multiple independent subsequence, and extracts key node in each subsequence, crucial joint
Point is reformulated new arrangement set and is carried out gesture identification.Reduce the temporal redundancy information of gesture feature, improve feature pair
The robustness of gesture velocity variations difference, the present invention solves techniques below problem: first, how to quantify gesture depth map sequence
Dependency between frame and frame；Second, how according to similarity, gesture sequence to be decomposed；3rd, the most after disassembly
Subsequence is extracted key node.
Gesture sequence, according to the dependency between depth map sequence, is decomposed by the embodiment of the present invention, and from decomposition
Subsequence in extract key point.Compared with additive method, the embodiment of the present invention has the advantage that first, based on direction
Gesture feature can fully describe the dependency between depth map frame and frame, and it is superfluous that the analysis of this dependency is conducive to removing time domain
Remaining information, improves the descriptive and discrimination of gesture feature.Second, gesture sequence decomposition algorithm based on spectral clustering can basis
Similar matrix, decomposes gesture sequence, and the result of decomposition is that similar frame is divided into same class, and dissimilar matrix is drawn
Assign to different classes.Gesture sequence is decomposed into multiple subprocess.Be conducive to overcoming between the gesture that velocity variations difference is brought
Diversity.3rd, key point extracting method based on minimum euclidean distance, from each subsequence, extract current subsequence
Representational image is as the key point of gesture.Set of keypoints eliminates temporal redundancy information, and overcomes velocity variations
The diversity of the same gesture caused.
Compared with other algorithms, the dynamic gesture that the gesture feature recognition methods of the present invention is decomposed in depth map sequence gesture
Discrimination is significantly improved during identification.Comparing result is as shown in Table 1 and Table 2.
The method of table 1 embodiment of the present invention contrasts at the discrimination of MSRGesture3D data base with prior art
| Algorithm | MSRGesture3D data base's discrimination (%) |
| Gesture feature recognition methods+the HON4D of the embodiment of the present invention | 90.59 |
| Uniform sampling+HON4D | 87.01 |
| HON4D [document 1] | 88.25 |
| Jiang et al. [document 2] | 88.50 |
| Yang et al. [document 3] | 89.20 |
| Klaser et al. [document 4] | 85.23 |
As shown in table 1, in common data sets MSRGesture3D data base, the gesture feature identification of the embodiment of the present invention
Method is compared with the gesture feature recognition methods of pertinent literature, and the method discrimination of the embodiment of the present invention is up to 90.59%, exceedes
The discrimination of the gesture feature recognition methods of prior art.
The method of table 2 embodiment of the present invention contrasts at the discrimination of self-defining data collection with prior art
As shown in table 2, in self-defining data storehouse, the gesture feature recognition methods of the embodiment of the present invention and pertinent literature
Gesture feature recognition methods is compared, and the method discrimination of the embodiment of the present invention is up to 85.76%, and the gesture exceeding prior art is special
Levy the discrimination of recognition methods.
Wherein, the pertinent literature in above table is as follows:
Document 1:Suk H I, Sin B K, Lee S W.Hand Gesture Recognition Based on
Dynamic Bayesian Network Framework.Pattern Recognition,2010,43(9):3059-3072.
Document 2: Feng Tong. gesture identification research based on neutral net: [Ph.D. Dissertation]. Beijing: Beijing University of Science & Engineering is big
Learn, 2015.
Document 3:Wenjun T, Chengdong W, Shuying Z etal.Dynamic Hand Gesture
Recognition Using Motion Trajectories and Key Frames.In:Proceedings of the
2nd IEEE International Conference On Advanced Computer Control(ICACC),2010,3:
163-167.
Document 4:Oreifej O, Liu Z.HON4D:Histogram of Oriented 4D Normals for
Activity Recognition from Depth Sequences.In:Proceedings of the IEEE
Conference On Computer Vision and Pattern Recognition,2013:716-723.
Document 5:Yang X, Zhang C, Tian Y L.Recognizing Actions Using Depth Motion
Maps-Based Histograms of Oriented Gradients.In:Proceedings of the 20th ACM
International Conference On Multimedia,2012,1057-1060.
Document 6:Tang S, Wang X, Lv X etal.Histogram of Oriented Normal Vectors
for Object Recognition with a Depth Sensor.In:Proceedings of Asian Conference
on Computer Vision.Springer Berlin Heidelberg,2012,525-538.
It should be noted that can be at such as one group of computer executable instructions in the step shown in the flow chart of accompanying drawing
Computer system performs, and, although show logical order in flow charts, but in some cases, can be with not
It is same as the step shown or described by order execution herein.
Embodiments providing a kind of gesture feature identification device, this gesture feature identification device may be used for performing
The gesture feature recognition methods of the embodiment of the present invention.
Fig. 2 is the schematic diagram of gesture feature identification device according to embodiments of the present invention, as in figure 2 it is shown, this device includes:
Computing unit 10, the similarity between frame and the frame calculating gesture depth map sequence, obtain result of calculation.
Resolving cell 20, for resolving into multiple subsequence according to result of calculation by gesture depth map sequence.
Extraction unit 30, for extracting the key node of each subsequence in multiple subsequence, wherein, key node is every
Individual subsequence meets pre-conditioned frame.
Assembled unit 40, for forming the set of keypoints of gesture depth map sequence by key node.
Recognition unit 50, for carrying out gesture identification according to set of keypoints, obtains gesture identification result.
Alternatively, computing unit 10 includes: decomposing module, for the image of depth map sequence is carried out piecemeal process,
To multiple image blocks；First computing module, for calculating the gradient direction angle of all pixels in each image block, by each
In image block, the maximum of gradient direction angle is as the orientation angle of each image block；Composite module, for all images
The orientation angle of block is elementary composition characteristic vector；Second computing module, for calculating gesture depth map sequence according to characteristic vector
Similarity between frame and the frame of row.
Alternatively, the second computing module includes: the first calculating sub module, for by Gauss similar function calculate feature to
The similar matrix of amount composition；Weighting submodule, for being weighted process, the similar matrix after being weighted to similar matrix；
Second calculating sub module, for according to weighting after similar matrix calculate gesture depth map sequence frame and frame between similar
Property.
Alternatively, resolving cell 20 includes: set up module, for setting up undirected artwork according to vertex set and adjacency matrix
Type, wherein, vertex set is the set of all depth maps in depth map sequence, any two two field pictures of element representation in adjacency matrix
Between similarity, adjacency matrix is made up of the similar matrix after weighting；Divide module, for graph model is divided, its
In, graph model is carried out division and includes: the similarity matrix after weighting is converted to Laplacian Matrix, solves Laplce's square
Battle array, obtains eigenvalue and characteristic vector；In characteristic vector second little characteristic vector is as optimal vector；To in optimal vector
Point clusters.
This embodiment uses computing unit 10 to calculate the similarity between frame and the frame of gesture depth map sequence, is calculated
Result；Gesture depth map sequence is resolved into multiple subsequence according to result of calculation by resolving cell 20；Extraction unit 30 extracts many
The key node of each subsequence in individual subsequence；Key node is formed the key point of gesture depth map sequence by assembled unit 40
Set；And recognition unit 50 carries out gesture identification according to set of keypoints, obtain gesture identification result, solve temporal redundancy
Information causes the descriptive low problem of gesture feature more, and then has reached to promote the illustrative effect of gesture feature.
Obviously, those skilled in the art should be understood that each module of the above-mentioned present invention or each step can be with general
Calculating device realize, they can concentrate on single calculating device, or be distributed in multiple calculating device and formed
Network on, alternatively, they can with calculate the executable program code of device realize, it is thus possible to by they store
Performed by calculating device in the storage device, or they are fabricated to respectively each integrated circuit modules, or by them
In multiple modules or step be fabricated to single integrated circuit module and realize.So, the present invention be not restricted to any specifically
Hardware and software combines.
The foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, for the skill of this area
For art personnel, the present invention can have various modifications and variations.All within the spirit and principles in the present invention, that is made any repaiies
Change, equivalent, improvement etc., should be included within the scope of the present invention.
Claims (10)
1. a gesture feature recognition methods, it is characterised in that including:
Calculate the similarity between frame and the frame of gesture depth map sequence, obtain result of calculation；
According to described result of calculation, described gesture depth map sequence is resolved into multiple subsequence；
Extracting the key node of each subsequence in the plurality of subsequence, wherein, described key node is described every sub-sequence
Row meet pre-conditioned frame；
Described key node is formed the set of keypoints of described gesture depth map sequence；And
Carry out gesture identification according to described set of keypoints, obtain gesture identification result.
Method the most according to claim 1, it is characterised in that calculate gesture depth map sequence frame and frame between similar
Property includes:
The image of described depth map sequence is carried out piecemeal process, obtains multiple image block；
Calculate the gradient direction angle of all pixels in each image block, by gradient direction angle in described each image block
Maximum is as the orientation angle of described each image block；
It is elementary composition characteristic vector with the orientation angle of all image blocks；
The similarity between frame and the frame of described gesture depth map sequence is calculated according to described characteristic vector.
Method the most according to claim 2, it is characterised in that calculate described gesture depth map sequence according to described characteristic vector
Similarity between frame and the frame of row includes:
The similar matrix of described characteristic vector composition is calculated by Gauss similar function；
It is weighted described similar matrix processing, the similar matrix after being weighted；
The similarity between frame and the frame of described gesture depth map sequence is calculated according to the similar matrix after described weighting.
Method the most according to claim 3, it is characterised in that according to described result of calculation by described gesture depth map sequence
Resolve into multiple subsequence to include:
Setting up non-directed graph model according to vertex set and adjacency matrix, wherein, described vertex set is all in described depth map sequence
The set of depth map, the similarity between any two two field pictures of element representation in described adjacency matrix, described adjacency matrix by
Similar matrix after described weighting is constituted；
Described graph model is divided, wherein, described graph model is carried out division and includes: by the similarity square after described weighting
Battle array is converted to Laplacian Matrix, solves described Laplacian Matrix, obtains eigenvalue and characteristic vector；With described characteristic vector
In the second little characteristic vector be optimal vector；Point in described optimal vector is clustered.
Method the most according to claim 4, it is characterised in that according to described result of calculation by described gesture depth map sequence
Resolve into multiple subsequence also to include:
Current depth graphic sequence is carried out two points of iteration clusters, wherein, current depth graphic sequence is carried out two points of iteration cluster bags
Include: the described optimal vector of current sequence being gathered is two classes, obtain two class optimal vectors, according to the similarity average of current sequence
With the condition that the relation of the similarity average of described two class optimal vectors determines iteration stopping.
Method the most according to claim 1, it is characterised in that extract the key of each subsequence in the plurality of subsequence
Node includes:
In described each subsequence, calculate the Euclidean distance between any two frames, obtain multiple Euclidean distance；
Using frame minimum for Euclidean distance described in each subsequence as the key node of described subsequence.
7. a gesture feature identification device, it is characterised in that including:
Computing unit, the similarity between frame and the frame calculating gesture depth map sequence, obtain result of calculation；
Resolving cell, for resolving into multiple subsequence according to described result of calculation by described gesture depth map sequence；
Extraction unit, for extracting the key node of each subsequence in the plurality of subsequence, wherein, described key node is
Described each subsequence meets pre-conditioned frame；
Assembled unit, for forming the set of keypoints of described gesture depth map sequence by described key node；And
Recognition unit, for carrying out gesture identification according to described set of keypoints, obtains gesture identification result.
Device the most according to claim 7, it is characterised in that described computing unit includes:
Decomposing module, for the image of described depth map sequence is carried out piecemeal process, obtains multiple image block；
First computing module, for calculating the gradient direction angle of all pixels in each image block, by described each image
In block, the maximum of gradient direction angle is as the orientation angle of described each image block；
Composite module, being used for the orientation angle of all image blocks is elementary composition characteristic vector；
Second computing module, be used between frame and the frame according to the described characteristic vector described gesture depth map sequence of calculating is similar
Property.
Device the most according to claim 8, it is characterised in that described second computing module includes:
First calculating sub module, for calculating the similar matrix of described characteristic vector composition by Gauss similar function；
Weighting submodule, for being weighted process, the similar matrix after being weighted to described similar matrix；
Second calculating sub module, for calculating frame and the frame of described gesture depth map sequence according to the similar matrix after described weighting
Between similarity.
Device the most according to claim 9, it is characterised in that described resolving cell includes:
Setting up module, for setting up non-directed graph model according to vertex set and adjacency matrix, wherein, described vertex set is the described degree of depth
The set of all depth maps in graphic sequence, the similarity between any two two field pictures of element representation in described adjacency matrix, institute
State adjacency matrix to be made up of the similar matrix after described weighting；
Divide module, for described graph model is divided, wherein, described graph model is carried out division and includes: add described
Similarity matrix after power is converted to Laplacian Matrix, solves described Laplacian Matrix, obtains eigenvalue and characteristic vector；
In described characteristic vector, the second little characteristic vector is as optimal vector；Point in described optimal vector is clustered.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610559968.7A CN106228121B (en) | 2016-07-15 | 2016-07-15 | Gesture feature recognition method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610559968.7A CN106228121B (en) | 2016-07-15 | 2016-07-15 | Gesture feature recognition method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106228121A true CN106228121A (en) | 2016-12-14 |
| CN106228121B CN106228121B (en) | 2019-12-06 |
Family
ID=57520628
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610559968.7A Active CN106228121B (en) | 2016-07-15 | 2016-07-15 | Gesture feature recognition method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106228121B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107212971A (en) * | 2017-07-11 | 2017-09-29 | 孟现华 | A kind of method for carrying aphasis patient |
| CN107260412A (en) * | 2017-07-11 | 2017-10-20 | 孟现华 | Folding intellectuality stretcher |
| CN107894834A (en) * | 2017-11-09 | 2018-04-10 | 上海交通大学 | Gesture identification method and system are controlled under augmented reality environment |
| CN109829422A (en) * | 2019-01-28 | 2019-05-31 | 哈尔滨工业大学 | A kind of video frequency identifying method based on the movement of impulsive neural networks falling over of human body |
| CN110176968A (en) * | 2019-05-20 | 2019-08-27 | 桂林理工大学 | A kind of hopping phenomenon correcting method in WiFi Human bodys' response |
| CN111125437A (en) * | 2019-12-24 | 2020-05-08 | 四川新网银行股份有限公司 | Method for identifying lip language picture in video |
| CN112527113A (en) * | 2020-12-09 | 2021-03-19 | 北京地平线信息技术有限公司 | Method and apparatus for training gesture recognition and gesture recognition network, medium, and device |
| CN113111842A (en) * | 2021-04-26 | 2021-07-13 | 浙江商汤科技开发有限公司 | Action recognition method, device, equipment and computer readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103593464A (en) * | 2013-11-25 | 2014-02-19 | 华中科技大学 | Video fingerprint detecting and video sequence matching method and system based on visual features |
| US20150248170A1 (en) * | 2013-07-12 | 2015-09-03 | Magic Leap, Inc. | Method and system for generating a virtual user interface related to a totem |
-
2016
- 2016-07-15 CN CN201610559968.7A patent/CN106228121B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150248170A1 (en) * | 2013-07-12 | 2015-09-03 | Magic Leap, Inc. | Method and system for generating a virtual user interface related to a totem |
| CN103593464A (en) * | 2013-11-25 | 2014-02-19 | 华中科技大学 | Video fingerprint detecting and video sequence matching method and system based on visual features |
Non-Patent Citations (2)
| Title |
|---|
| 余旭: "基于Kinect传感器的动态手势识别", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 高智等: "基于梯度方向角量化的匹配新算法", 《计算机工程》 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107260412B (en) * | 2017-07-11 | 2019-01-04 | 雷磊 | Folding intelligence stretcher |
| CN107260412A (en) * | 2017-07-11 | 2017-10-20 | 孟现华 | Folding intellectuality stretcher |
| CN107212971A (en) * | 2017-07-11 | 2017-09-29 | 孟现华 | A kind of method for carrying aphasis patient |
| CN107212971B (en) * | 2017-07-11 | 2018-09-21 | 薛红 | A kind of folding intelligence stretcher |
| CN107894834B (en) * | 2017-11-09 | 2021-04-02 | 上海交通大学 | Control gesture recognition method and system in augmented reality environment |
| CN107894834A (en) * | 2017-11-09 | 2018-04-10 | 上海交通大学 | Gesture identification method and system are controlled under augmented reality environment |
| CN109829422A (en) * | 2019-01-28 | 2019-05-31 | 哈尔滨工业大学 | A kind of video frequency identifying method based on the movement of impulsive neural networks falling over of human body |
| CN110176968A (en) * | 2019-05-20 | 2019-08-27 | 桂林理工大学 | A kind of hopping phenomenon correcting method in WiFi Human bodys' response |
| CN110176968B (en) * | 2019-05-20 | 2021-04-06 | 桂林理工大学 | Jump phenomenon correction method for WiFi human behavior recognition |
| CN111125437A (en) * | 2019-12-24 | 2020-05-08 | 四川新网银行股份有限公司 | Method for identifying lip language picture in video |
| CN111125437B (en) * | 2019-12-24 | 2023-06-09 | 四川新网银行股份有限公司 | Method for recognizing lip language picture in video |
| CN112527113A (en) * | 2020-12-09 | 2021-03-19 | 北京地平线信息技术有限公司 | Method and apparatus for training gesture recognition and gesture recognition network, medium, and device |
| CN113111842A (en) * | 2021-04-26 | 2021-07-13 | 浙江商汤科技开发有限公司 | Action recognition method, device, equipment and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106228121B (en) | 2019-12-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106228121A (en) | Gesture feature recognition methods and device | |
| CN111814719B (en) | Skeleton behavior recognition method based on 3D space-time diagram convolution | |
| CN106778604B (en) | Pedestrian re-identification method based on matching convolutional neural network | |
| CN103839065B (en) | Extraction method for dynamic crowd gathering characteristics | |
| CN109344874A (en) | A kind of automatic chromosome analysis method and system based on deep learning | |
| Risojević et al. | Gabor descriptors for aerial image classification | |
| CN103258324B (en) | Based on the method for detecting change of remote sensing image that controlled kernel regression and super-pixel are split | |
| CN110781766B (en) | Grassman manifold discriminant analysis image recognition method based on characteristic spectrum regularization | |
| CN109034035A (en) | Pedestrian's recognition methods again based on conspicuousness detection and Fusion Features | |
| CN102708370A (en) | Method and device for extracting multi-view angle image foreground target | |
| CN105825233B (en) | A kind of pedestrian detection method based on on-line study random fern classifier | |
| CN107633226A (en) | A kind of human action Tracking Recognition method and system | |
| CN107527054A (en) | Prospect extraction method based on various visual angles fusion | |
| CN103824062B (en) | Motion identification method for human body by parts based on non-negative matrix factorization | |
| CN102592150B (en) | Gait identification method of bidirectional two-dimensional principal component analysis based on fuzzy decision theory | |
| CN103020614A (en) | Human movement identification method based on spatio-temporal interest point detection | |
| CN103413154A (en) | Human motion identification method based on normalized class Google measurement matrix | |
| CN103714340A (en) | Self-adaptation feature extracting method based on image partitioning | |
| CN105718934A (en) | Method for pest image feature learning and identification based on low-rank sparse coding technology | |
| Shi et al. | Remote sensing scene classification based on multibranch fusion attention network | |
| KR100956747B1 (en) | Computer Architecture Combining Neural Network and Parallel Processor, and Processing Method Using It | |
| CN104008374B (en) | Miner's detection method based on condition random field in a kind of mine image | |
| CN104778683B (en) | A kind of multi-modality images dividing method based on Functional Mapping | |
| CN111127407B (en) | Fourier transform-based style migration forged image detection device and method | |
| CN115115847B (en) | Three-dimensional sparse reconstruction method and device and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |