CN108392207B - Gesture tag-based action recognition method - Google Patents

Gesture tag-based action recognition method Download PDF

Info

Publication number
CN108392207B
CN108392207B CN201810133363.0A CN201810133363A CN108392207B CN 108392207 B CN108392207 B CN 108392207B CN 201810133363 A CN201810133363 A CN 201810133363A CN 108392207 B CN108392207 B CN 108392207B
Authority
CN
China
Prior art keywords
label
key node
frame
attitude
tag
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201810133363.0A
Other languages
Chinese (zh)
Other versions
CN108392207A (en
Inventor
徐嘉晨
张晓云
刘小通
周建益
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northwestern University
Original Assignee
Northwestern University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northwestern University filed Critical Northwestern University
Priority to CN201810133363.0A priority Critical patent/CN108392207B/en
Publication of CN108392207A publication Critical patent/CN108392207A/en
Application granted granted Critical
Publication of CN108392207B publication Critical patent/CN108392207B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1121—Determining geometric values, e.g. centre of rotation or angular range of movement
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1126—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique
    • A61B5/1128—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique using image analysis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Surgery (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biomedical Technology (AREA)
  • Pathology (AREA)
  • Public Health (AREA)
  • Physiology (AREA)
  • Veterinary Medicine (AREA)
  • Dentistry (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Geometry (AREA)
  • Psychiatry (AREA)
  • Signal Processing (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a gesture tag-based action recognition method, which abstracts action recognition into gesture recognition, abstracts gestures into gesture tags based on a key node relative position method, and finds out actions sent by human beings by comparing gesture changes of the human beings for a certain time; the method reduces the difficulty of establishing the template library, greatly reduces the speed and the operation requirement of the action recognition, and improves the universality of the action recognition on the individual recognition. The method has important application value in the fields of human-computer interaction, virtual reality, video monitoring and motion characteristic analysis.

Description

Gesture tag-based action recognition method
Technical Field
The invention belongs to the technical field of motion recognition, and relates to a motion recognition method based on a posture label.
Background
Action recognition is a hot point of research in recent years, and research results generated in the existing action recognition field are applied to various fields such as people's air defense security, human life habit research, man-machine interaction, virtual reality and the like, and have great positive effects. In the traditional action recognition, an image (including a video, a plurality of photos and the like) is directly analyzed through a technical method related to image processing, and the action recognition is finally realized through the steps of image segmentation, feature extraction, action feature classification and the like. Although the existing action recognition method is greatly developed, certain problems still exist, such as huge calculation amount; the action characteristic library is not well established, and a professional is required to input materials; when people with body heights different from the body height of the material are identified, the precision is greatly reduced, and the like.
Disclosure of Invention
Aiming at the problems in the prior art, the invention aims to provide an action recognition method based on attitude tags, which solves the problems of large calculation amount, difficult establishment of a template library and poor universality of the template library in the prior action recognition technology.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method of decomposing an action into gesture tags, comprising the steps of:
step 1, adopting skeleton tracking equipment to obtain position data of key nodes of human trunk actions at each moment, wherein the position data of the key nodes are data in a skeleton tracking equipment coordinate system; the key nodes at least comprise a key node HEAD, a key node SHOULDER CENTER, a key node SPINE, a key node HIP CENTER, a key node SHOULDE RIGHT, a key node SHOULDER LEFT, a key node ELBOW RIGHT, a key node ELBOW LEFT, a key node WRIST RIGHT, a key node WRIST LEFT, a key node HAND RIGHT, a key node HANDLEFT, a key node HIPRIGHT, a key node HIPLEFT, a key node KNIEE RIGHT, a key node KNIEE LEFT, a key node ANIKLE RIGHT, a key node ANIKLE LEFT, a key node FOOT RIGHT and a key node FOOT LEFT;
step 2, respectively converting the position data of the key nodes at each moment obtained in the step 1 into position data of the key nodes in a morphological coordinate system; the morphological coordinate system takes the facing direction of the human body trunk as the positive direction of a Z axis, the direction of the morphological upper end of the human body trunk as the positive direction of a Y axis, the left direction of the human body as the positive direction of an X axis and the key node HIP CENTER as the origin;
step 3, respectively obtaining attitude labels at each moment by using the position data of the key nodes under the morphological coordinate system at each moment, which are obtained in the step 2, wherein the attitude labels comprise main body attitude labels GLbodyLeft forelimb posture label GLlfRight front limb posture label GLrfLeft hind limb posture label GLlbAnd right hind limb posture label GLrb。
Optionally, the subject posture label GL in step 3bodyThe method for obtaining (1) is as follows:
selecting XF，YFAnd ZFFinding the coordinate value with the maximum absolute value, and finding the GL corresponding to the interval to which the coordinate value belongsbodyIs the subject posture label GLbodyThe following formula is adopted:
Figure GDA0002742446610000021
wherein, XF，YFAnd ZFCoordinates of 3 coordinate axes of the unit vector F respectively; unit vector
Figure GDA0002742446610000022
Figure GDA0002742446610000031
Figure GDA0002742446610000032
A vector formed by the key node HEAD and the key node HIP CENTER in the skeleton tracking device coordinate system;
the left forelimb posture label GL in the step 3lfRight front limb posture label GLrfLeft hind limb posture label GLlbAnd right hind limb posture label GLrbThe method for obtaining (1) is as follows:
the four posture labels all comprise three key nodes which are marked as a key node 1, a key node 2 and a key node 3, and the posture label GL of the left forelimblfThree key nodes included are ELBOW LEFT, WRIST LEFT and HAND LEFT, respectively, for the right forelimb posture label GLrfThree key nodes included are KNIEE LEFT, ANIKLE LEFT and FOOT LEFT, respectively, for LEFT hind limb posture label GLlbThe three key nodes are ELBOW LEFT, WRIST LEFT and HAND LEFT, and the posture mark of right hind limbLabel GLrbThe three key nodes included are KNIEE LEFT, ANIKLE LEFT and FOOT LEFT, respectively.
The data of the three key nodes in the morphological coordinate system are respectively used as (X)1,Y1,Z1)(X2,Y2,Z2)(X3,Y3,Z3) Represents; the four attitude tags each include height tag G1, orientation tag G2, and curl tag G3;
the height label G1 is obtained by the following method:
G1＝(g1+g2+g3) And/3, rounding, wherein,
Figure GDA0002742446610000033
wherein n is 1,2,3, YHIs a Y-axis coordinate in the morphological coordinate system of the key node HEADHCThe Y-axis coordinate of the key node SHOULDER CENTER under the morphological coordinate system;
the orientation label G2 is obtained as follows:
counting the symbols of the X-axis coordinate and the Z-axis coordinate of the key node 1, the key node 2 and the key node 3, and solving an orientation label G2 by adopting the following formula:
Figure GDA0002742446610000041
the method for obtaining the crimp label G3 is as follows:
introducing a key node 4 according to the key node 1, the key node 2 and the key node 3, respectively calculating the distances D between the key node 1, the key node 2 and the key node 3 and the key node 41，D2，D3(ii) a For left forelimb posture label GLlfThe key node 4 is SHOULDER LEFT, for the right forelimb posture label GLrfThe key node 4 is SHOULDER RIGHT, for left hind limb posture label GLlbThe key node 4 is HIPLEFT, and the right hind limb posture label GLrbThe key node 4 is HIPRIGHT；
The value of the curl label G3 is given by the following formula:
Figure GDA0002742446610000042
the invention also provides a method for obtaining the action template library, which comprises the following steps:
step 1, making standard actions for multiple times, and decomposing the standard actions made each time into attitude tags at each moment; selecting the attitude tag at the initial moment as an initial frame attitude tag, and selecting the attitude tag at the termination moment as a termination frame attitude tag; taking the standard action made for the first time as a comparison standard action, and taking the standard actions made for other times as reference standard actions; comparing the initial frame posture label corresponding to the standard action as an initial frame comparison posture label, and using the termination frame posture label corresponding to the first made standard action as a termination frame comparison posture label;
the standard action made each time is decomposed into attitude tags at various moments, wherein the attitude tags are obtained according to the method of claim 1;
step 2, solving an initial frame similarity coefficient group, wherein the specific method is as follows:
respectively calculating the similarity Sl1(A) of each attribute of the start frame attitude tag and the start frame contrast attitude tags of a plurality of reference standard actionsnThe formula used is as follows:
Sl1(A)n＝An×z1n÷ln(n∈Z，n∈[1，13]when n is 1,4,7,10,13, ln is 5, and the rest is 3)
Wherein A isnFor initialized similarity coefficient values, AnN denotes the serial number of the attribute, and the serial numbers 1 to 13 are the body posture labels GL of the posture labels, respectivelybodyLeft forelimb posture label GLlfHeight label G1 in (1), left forelimb posture label GLlfMiddle orientation label G2, left forelimb posture label GLlfMiddle curl tag G3, left hind limb posture tag GLlbHeight label G1 in (1), left hind limb posture labelLabel GLlbMiddle orientation label G2, left hind limb posture label GLlbMiddle curl label G3, right front limb posture label GLrfHeight label G1, right front limb posture label GLrfOrientation label G2, right forelimb posture label GLrfThe curl label G3, the right hind limb posture label GLrbHeight label G1, right hind limb posture label GLrbOrientation label G2, right hind limb posture label GLrbCrimp label G3; z1nThe absolute value of the difference value of the corresponding attributes of the initial frame attitude tag and the initial frame contrast attitude tag of the reference standard action;
for each attribute n, selecting similarity Sl1(A) calculated according to the starting frame attitude labels of a plurality of reference standard actionsnThe second largest value in (b) is taken as the similarity coefficient value A1 under the attributen. The similarity coefficient value A1 corresponding to each attribute nnForming an initial frame similarity coefficient set Astar＝{A1n,n∈Z,n＝1,2,...,13}；
And step 3, solving a similarity coefficient group of the termination frame, wherein the specific method comprises the following steps:
respectively calculating the similarity Sl2(A) of each attribute of the terminal frame attitude tag and the terminal frame contrast attitude tags of a plurality of reference standard actionsnThe formula used is as follows:
Sl2(A)n＝An×z2n÷ln(n∈Z，n∈[1，13]when n is 1,4,7,10,13, ln is 5, and the rest is 3)
Wherein, z2nThe absolute value of the difference between the corresponding attributes of the terminating frame pose tag and the terminating frame contrast pose tag for the reference standard action;
selecting similarity Sl2(A) calculated according to the attitude labels of the termination frames of a plurality of reference standard actions for each attribute nnThe second largest value in (b) is taken as the similarity coefficient value A2 under the attributen(ii) a The similarity coefficient value A2 corresponding to each attribute nnForming a group A of termination frame similarity coefficientsstop＝{A2n,n∈Z,n＝1,2,...,13}；
And 4, aiming at a plurality of standard actions, obtaining an initial frame similarity coefficient group and an ending frame similarity coefficient group corresponding to each standard action according to the method in the step 1-3, wherein the initial frame similarity coefficient group and the ending frame similarity coefficient group corresponding to all the standard actions form an action template library.
The invention also provides an action recognition method based on the attitude tag, which comprises the following steps:
step 1, decomposing the action to be recognized into attitude tags at each moment aiming at the action to be recognized; the gesture label for decomposing the action to be recognized into each moment is obtained according to the method of claim 1;
step 2, selecting a certain standard action in the action template library, calculating the similarity SL (B) of each attribute between the termination frame attitude tag obtained in the step 1 and the termination frame attitude tag of the selected standard actionnAnd recording the attitude tag of the termination frame as the attitude tag of the tth frame, wherein the adopted formula is as follows:
SL(B)n＝A1n×z3n÷ln(n∈Z，n∈[1，13]when n is 1,4,7,10,13, ln is 5, and the rest is 3)
Wherein, z3nThe absolute value of the difference value of the corresponding attributes of the termination frame attitude tag obtained in the step 1 and the termination frame attitude tag of the selected standard action;
calculating the overall similarity S (B) between the attitude tag of the termination frame obtained in the step 1 and the attitude tag of the termination frame of the selected standard action, wherein the formula is as follows:
Figure GDA0002742446610000061
step 3, if the overall similarity S (B) is greater than the set threshold MAXBLUR, returning to the step 2; otherwise, executing step 4;
step 4, calculating the similarity SL (C) of each attribute between the previous frame attitude tag of the ending frame attitude tag and the starting frame attitude tag of the selected standard actionnAnd recording a frame attitude tag as a t-1 frame attitude tag, wherein the adopted formula is as follows:
SL(C)n＝A2n×z4n÷ln(n∈Z，n∈[1，13]when n is 1,4,7,10,13, ln is 5, and the rest is 3)
Wherein, z4nThe absolute value of the difference value of the corresponding attributes of the previous frame attitude tag and the selected initial frame attitude tag of the standard action is obtained;
calculating the overall similarity S (C) between the pose label of the last frame and the pose label of the initial frame of the selected standard action by the following formula:
Figure GDA0002742446610000071
step 5, if the overall similarity S (C) is less than a set threshold MAXBLUR, the action to be recognized is consistent with the selected standard action; and if the overall similarity S (C) is greater than the set threshold MAXBLUR, returning to the step 4, replacing the posture label of the t-1 frame of the processed object with the posture label of the t-2 frame of the processed object until the overall similarity S (C) is greater than the set threshold MAXBLUR when the processed object is the posture label of the first frame, and returning to the step 2.
Compared with the prior art, the invention has the following technical effects: the method abstracts action recognition into gesture recognition, abstracts gestures into gesture tags based on a key node relative position method, and finds out actions sent by human beings by comparing gesture changes of the human beings for a certain time; the method reduces the difficulty of establishing the template library, greatly reduces the speed and the operation requirement of the action recognition, and improves the universality of the action recognition on the individual recognition. The method has important application value in the fields of human-computer interaction, virtual reality, video monitoring and motion characteristic analysis.
The invention will be explained and explained in more detail below with reference to the figures and exemplary embodiments.
Drawings
FIG. 1 is a schematic representation of a skeletal tracking device coordinate system used in the present invention.
Fig. 2 is a schematic diagram of twenty key bone node positions obtained by the present invention.
Detailed Description
The invention provides a method for decomposing actions into attitude tags, which comprises the following steps:
step 1, adopting skeleton tracking equipment to obtain position data of key nodes of human trunk actions, wherein the position data of the key nodes are data in a skeleton tracking equipment coordinate system. The bone tracking device can adopt Kinect, key node data of actions are obtained by adopting the Kinect according to a certain frequency, the position data of the key nodes represent the positions of twenty specific bone nodes of bones, and the node names and the serial numbers of the key nodes are shown in the following table:
Figure GDA0002742446610000081
Figure GDA0002742446610000091
the skeleton tracking equipment coordinate system takes an equipment camera as an original point, the direction opposite to the camera is the positive direction of a Z axis, the reverse direction of gravity is the positive direction of a Y axis, the left direction of the camera is the positive direction of an X axis, and the unit length is 1 meter. The skeletal tracking device coordinate system is a static coordinate system.
Step 2, respectively converting the position data of the key nodes at each moment obtained in the step 1 into position data of the key nodes in a morphological coordinate system; the formula used is as follows:
Figure GDA0002742446610000092
wherein (X, y, z) ═ X-XHC，Y-YHC，Z-ZHC) Coordinates of vectors between any key NODEs NODE in the skeleton tracking equipment coordinate system obtained in the step 1 are shown, (X, Y, Z) shows position data of the key NODEs NODE, (XHC，YHC，ZHC) Location data representing key node HIPCENTER; alpha, beta and gamma are in the morphological coordinate system respectivelyThe rotation angle of each coordinate axis relative to the skeletal tracking device coordinate system.
The position data of the key node in the morphological coordinate system is (x ', y ', z ').
The morphological coordinate system uses the facing direction of the human trunk as the positive direction of the Z axis, the direction of the morphological upper end of the human trunk as the positive direction of the Y axis, the left direction of the human as the positive direction of the X axis, and the key node HIP CENTER as the origin.
The morphological upper end of the human body trunk refers to the morphological upper end of the part which reaches earlier, that is, the part which reaches later, with the head of the human body as the starting point, and extends downwards and outwards along the body. For example, when a person stands upright, the hands of the person naturally hang down, and the three parts of the left shoulder, the left elbow and the left hand are as follows: the left shoulder is the upper morphological end of the left elbow, which is the upper morphological end of the left hand.
Step 3, solving the main body posture label GL at each momentbodyLeft forelimb posture label GLlfRight front limb posture label GLrfLeft hind limb posture label GLlbAnd right hind limb posture label GLrb。
Specifically, in still another embodiment, the determination method of the facing direction of the human trunk and the morphological upper end direction of the human trunk in step 2 is as follows:
the position data of the key node SHOULDER RIGHT obtained in the step 1 is (X)SR，YSR，ZSR) And the position data of the key node SHOULDER LEFT is (X)SL，YSL，ZSL) The position data of the key node HIP CENTER is (X)HC，YHC，ZHC) The three key nodes can determine a plane, which is the plane of the human body.
Normal vector of human body trunk plane
Figure GDA0002742446610000101
Wherein,
Figure GDA0002742446610000102
Figure GDA0002742446610000103
Figure GDA0002742446610000104
calculating the vector of the key node HEAD and the key node HIP CENTER in the coordinate system of the skeletal tracking device
Figure GDA0002742446610000105
Since the head of the Kinect equipment always inclines forwards, the Kinect equipment will be
Figure GDA0002742446610000106
Multiplication by
Figure GDA0002742446610000107
If the value is positive, the control circuit is switched to the normal state,
Figure GDA0002742446610000108
taking a positive sign, if the value is negative,
Figure GDA0002742446610000109
taking the negative sign.
Figure GDA00027424466100001010
The direction of (a) is the human body trunk facing direction,
Figure GDA00027424466100001011
the direction of (a) is the morphological upper end direction of the human body trunk.
Specifically, the body posture label GLbodyThe method for obtaining (1) is as follows:
calculating the vector of the key node HEAD and the key node HIP CENTER in the coordinate system of the skeletal tracking device
Figure GDA00027424466100001012
Let unit vector
Figure GDA00027424466100001013
Selecting XF，YFAnd ZFFinding the coordinate value with the maximum absolute value, and finding the GL corresponding to the interval to which the coordinate value belongsbodyIs the subject posture label GLbodyThe following formula is adopted:
Figure GDA0002742446610000111
since F is a unit vector, then
Figure GDA0002742446610000112
XF，YFAnd ZFOne of the two values is 0, and when the other two values are equal, the two values which are equal are obtained as
Figure GDA0002742446610000113
Then XF，YFAnd ZFIs greater than
Figure GDA0002742446610000114
Left forelimb posture label GLlfRight front limb posture label GLrfLeft hind limb posture label GLlbAnd right hind limb posture label GLrbThe method for obtaining (1) is as follows:
the four posture labels all comprise three key nodes which are marked as a key node 1, a key node 2 and a key node 3, and the posture label GL of the left forelimblfThree key nodes included are ELBOW LEFT, WRIST LEFT and HAND LEFT, respectively, for the right forelimb posture label GLrfThree key nodes included are KNIEE LEFT, ANIKLE LEFT and FOOT LEFT, respectively, for LEFT hind limb posture label GLlbThe three key nodes are ELBOW LEFT, WRIST LEFT and HAND LEFT, right hind limb posture label GLrbThe three key nodes are KNIEE LEFT, ANIKLE LEFT and FOOT LEFT。
The data of the three key nodes in the morphological coordinate system are respectively used as (X)1,Y1,Z1)(X2,Y2,Z2)(X3,Y3,Z3) Represents; the four attitude tags each include height tag G1, orientation tag G2, and curl tag G3.
The height label G1 is obtained by the following method:
G1＝(g1+g2+g3) The rounding of/3, the smaller the value of G1, indicates that the position of the part is closer to the upper end of morphology. Wherein:
Figure GDA0002742446610000121
wherein n is 1,2,3, YHIs a Y-axis coordinate in the morphological coordinate system of the key node HEADHCIs the Y-axis coordinate of the key node SHOULDER CENTER in the morphological coordinate system, and Y isH＞YHC。
The orientation label G2 is obtained as follows:
counting the symbols of the X-axis coordinate and the Z-axis coordinate of the key node 1, the key node 2 and the key node 3, and solving an orientation label G2 by adopting the following formula:
Figure GDA0002742446610000122
the method for obtaining the crimp label G3 is as follows:
introducing a key node 4 according to the key node 1, the key node 2 and the key node 3, respectively calculating the distances D between the key node 1, the key node 2 and the key node 3 and the key node 41，D2，D3. For left forelimb posture label GLlfThe key node 4 is SHOULDER LEFT, for the right forelimb posture label GLrfThe key node 4 is SHOULDER RIGHT, for left hind limb posture label GLlbThe key node 4 is HIPLEFT, and the right hind limb posture label GLrbAnd the key node 4 is HIPRIGHT.
The value of the curl label G3 is given by the following formula:
Figure GDA0002742446610000123
another aspect of the present invention provides a method for obtaining an action template library, including the following steps:
step 1, making standard actions for multiple times, and decomposing the standard actions made each time into attitude tags at various moments according to the method for decomposing the actions into the attitude tags; and selecting the attitude tag at the initial moment as an initial frame attitude tag, and selecting the attitude tag at the termination moment as a termination frame attitude tag. And taking the standard action made for the first time as a comparison standard action, and taking the standard actions made for other times as reference standard actions. And comparing the initial frame posture label corresponding to the standard action as an initial frame comparison posture label, and using the termination frame posture label corresponding to the first-made standard action as a termination frame comparison posture label.
Step 2, solving an initial frame similarity coefficient group, wherein the specific method is as follows:
respectively calculating the similarity Sl1(A) of each attribute of the start frame attitude tag and the start frame contrast attitude tags of a plurality of reference standard actionsnThe formula used is as follows:
Sl1(A)n＝An×z1n÷ln(n∈Z，n∈[1，13]when n is 1,4,7,10,13, ln is 5, and the rest is 3) (6)
Wherein A isnFor initialized similarity coefficient values, AnN denotes the serial number of the attribute, and the serial numbers 1 to 13 are the body posture labels GL of the posture labels, respectivelybodyLeft forelimb posture label GLlfHeight label G1 in (1), left forelimb posture label GLlfMiddle orientation label G2, left forelimb posture label GLlfMiddle curl tag G3, left hind limb posture tag GLlbHeight label G1 in, left hind limb posture label GLlbMiddle orientation label G2, left hind limb posture labelLabel GLlbMiddle curl label G3, right front limb posture label GLrfHeight label G1, right front limb posture label GLrfOrientation label G2, right forelimb posture label GLrfThe curl label G3, the right hind limb posture label GLrbHeight label G1, right hind limb posture label GLrbOrientation label G2, right hind limb posture label GLrbCrimp label G3; z1nAbsolute value of difference of start frame pose tag and corresponding attribute of start frame contrast pose tag for reference standard action, e.g. z11Subject attitude tag GL in start frame attitude tag for referencing standard actionbodySubject pose tag GL comparing pose tags with start framebodyThe absolute value of the difference of (a).
For each attribute n, selecting similarity Sl1(A) calculated according to the starting frame attitude labels of a plurality of reference standard actionsnThe second largest value in (b) is taken as the similarity coefficient value A1 under the attributen. The similarity coefficient value A1 corresponding to each attribute nnForming an initial frame similarity coefficient set Astar＝{A1n,n∈Z,n＝1,2,...,13}。
And step 3, solving a similarity coefficient group of the termination frame, wherein the specific method comprises the following steps:
respectively calculating the similarity Sl2(A) of each attribute of the terminal frame attitude tag and the terminal frame contrast attitude tags of a plurality of reference standard actionsnThe formula adopted is as follows;
Sl2(A)n＝An×z2n÷ln(n∈Z，n∈[1，13]when n is 1,4,7,10,13, ln is 5, and the rest is 3) (7)
Wherein, z2nAbsolute value of difference of corresponding attribute of terminating frame pose tag and terminating frame contrast pose tag for reference standard action, e.g. z21Subject pose tag GL in termination frame pose tag for referencing standard actionsbodySubject pose tag GL comparing pose tags with termination framesbodyThe absolute value of the difference of (a).
Selecting similarity Sl2(A) calculated according to the attitude labels of the termination frames of a plurality of reference standard actions for each attribute nnThe second largest value in (b) is taken as the similarity coefficient value A2 under the attributen. The similarity coefficient value A2 corresponding to each attribute nnForming a group A of termination frame similarity coefficientsstop＝{A2n,n∈Z,n＝1,2,...,13}。
And 4, aiming at a plurality of standard actions, obtaining an initial frame similarity coefficient group and an ending frame similarity coefficient group corresponding to each standard action according to the method in the step 1-3, wherein the initial frame similarity coefficient group and the ending frame similarity coefficient group corresponding to all the standard actions form an action template library.
A third aspect of the present invention provides a motion recognition method, including the steps of:
step 1, aiming at the action to be recognized, the action to be recognized is decomposed into the attitude tags at each moment according to the method for decomposing the action into the attitude tags.
Step 2, selecting a certain standard action in the action template library, calculating the similarity SL (B) of each attribute between the termination frame attitude tag obtained in the step 1 and the termination frame attitude tag of the selected standard actionnAnd recording the attitude tag of the termination frame as the attitude tag of the tth frame, wherein the adopted formula is as follows:
SL(B)n＝A1n×z3n÷ln(n∈Z，n∈[1，13]when n is 1,4,7,10,13, ln is 5, and the rest is 3)
(8)
Wherein, z3nAnd (4) obtaining the absolute value of the difference value of the corresponding attributes of the termination frame attitude tag obtained in the step (1) and the termination frame attitude tag of the selected standard action.
Calculating the overall similarity S (B) between the attitude tag of the termination frame obtained in the step 1 and the attitude tag of the termination frame of the selected standard action, wherein the formula is as follows:
Figure GDA0002742446610000151
step 3, if the overall similarity S (B) is greater than the set threshold MAXBLUR, returning to the step 2; otherwise, executing step 4;
step 4, calculating the similarity SL (C) of each attribute between the previous frame attitude tag of the ending frame attitude tag and the starting frame attitude tag of the selected standard actionnAnd recording a frame attitude tag as a t-1 frame attitude tag, wherein the adopted formula is as follows:
SL(C)n＝A2n×z4n÷ln(n∈Z，n∈[1，13]when n is 1,4,7,10,13, ln is 5, and the rest is 3) (10)
Wherein, z4nThe absolute value of the difference value of the corresponding attributes of the previous frame attitude tag and the selected initial frame attitude tag of the standard action.
Calculating the overall similarity S (C) between the pose label of the last frame and the pose label of the initial frame of the selected standard action by the following formula:
Figure GDA0002742446610000152
step 5, if the overall similarity S (C) is less than a set threshold MAXBLUR, the action to be recognized is consistent with the selected standard action; and if the overall similarity S (C) is greater than the set threshold MAXBLUR, returning to the step 4, replacing the posture label of the t-1 frame of the processed object with the posture label of the t-2 frame of the processed object until the overall similarity S (C) is greater than the set threshold MAXBLUR when the processed object is the posture label of the first frame, and returning to the step 2. MAXBLUR represents the fuzzy degree of the action matching algorithm, and the value is 0.25-0.05.
Examples
And (3) performing action recognition by adopting a traditional method:
the using equipment is a single Kinect, when the identification action is right hand saluting, a template library is established by using a traditional method, and the height of a tester a is 173cm, the weight of the tester a is 60kg, the height of a tester b is 191cm, the weight of the tester b is 100kg, the height of a tester c is 181cm, and the weight of the tester c is 80 kg. The first 50 samples are recorded for a tester a, the samples with the sample serial numbers of 51-80 are recorded for a tester b, the time of recording the samples is about 2 minutes each time, the recorded action is selected during recording, the recorder takes right hand salutation action when standing 1.5 meters in front of the device, the sample library and the test points are all 20 skeleton nodes of Kinect
During testing, a tester is specified to stand 1.5 meters in front of the equipment, standard-meeting actions are performed as far as possible, each tester performs right hand salutation actions ten times when a new sample optimization template library is input, and recognition results are counted. The recognition statistics are shown in table 1:
TABLE 1
Number of samples 20 30 40 50 60 70 80
Tester a 70％ 80％ 90％ 100％ 90％ 80％ 80％
Tester b 0％ 30％ 30％ 30％ 70％ 70％ 90％
Tester c 30％ 40％ 40％ 40％ 50％ 50％ 50％
From the test results it can be seen that: when the tester a is used as a tester and is used as a sample logger, the recognition success rate is obviously increased along with the increase of logging times, the final recognition success rate reaches 100% when the number of samples is 50, the success rates of other testers are basically unchanged, when the tester b is used as the sample logger, the success rate of the tester b is greatly improved, and the success rate of the tester a is reduced. The tester c has a low recognition success rate because it is not involved in the entry, but the success rate increases as the number of samples increases. Test 1 took a total of 4 hours and 20 minutes.
The method of the invention is used for action recognition:
the used equipment is a single Kinect, the recognized actions are right hand salutation and double hand waving, the method is used for establishing an attitude tag library, an action-attitude library, all 20 nodes are used, and the total time consumed for establishing the template library is 30 minutes, and the method comprises six actions: standing, left hand high lifting, right hand high lifting, both hands high lifting, left arm saluting and right arm saluting. Wherein the right hand high lift is similar to the right arm salute action, the double hand high lift is a complex action of three postures, the action not only meets the left hand high lift requirement, but also meets the right hand high lift requirement, and the test is added for increasing the test difficulty. Tester a was 173cm in height and 60kg in weight, tester b was 191cm in height and 100kg in weight, and tester c was 181cm in height and 80kg in weight, in accordance with test 1.
During testing, a tester is specified to stand 1.5 meters ahead of the device and make standard-meeting actions as far as possible, each person makes right hand salutation and two-hand waving actions ten times respectively, and the template library is not updated in the whole test, so that each person does not need to make multiple rounds of actions and count recognition results. The recognition statistics are shown in table 2:
TABLE 2
Figure GDA0002742446610000171
Wherein the tester c mistakenly recognizes one right arm salute action as right-hand high lift, and one both-hand high lift action as right-hand high lift, in relation to the relevant action settings in the action-posture library.
The overall test has a much higher recognition success rate than test 1, and has a good success rate for three testers with different body types. The overall test takes 1 hour and 10 minutes in total, and the recognition action is richer and more difficult than the test 1.
The method has good universality for testers, and the input (design) of the template library is simpler and more convenient.

Claims (3)

1. A method of decomposing an action into gesture tags, comprising the steps of:
step 1, adopting skeleton tracking equipment to obtain position data of key nodes of human trunk actions at each moment, wherein the position data of the key nodes are data in a skeleton tracking equipment coordinate system; the key nodes at least comprise a key node HEAD, a key node SHOULDER CENTER, a key node SPINE, a key node HIP CENTER, a key node SHOULDE RIGHT, a key node SHOULDER LEFT, a key node ELBOW RIGHT, a key node ELBOW LEFT, a key node WRIST RIGHT, a key node WRIST LEFT, a key node HAND RIGHT, a key node HANDLEFT, a key node HIPRIGHT, a key node HIPLEFT, a key node KNIEE RIGHT, a key node KNIEE LEFT, a key node ANIKLE RIGHT, a key node ANIKLE LEFT, a key node FOOT RIGHT and a key node FOOT LEFT;
step 2, respectively converting the position data of the key nodes at each moment obtained in the step 1 into position data of the key nodes in a morphological coordinate system; the morphological coordinate system takes the facing direction of the human body trunk as the positive direction of a Z axis, the direction of the morphological upper end of the human body trunk as the positive direction of a Y axis, the left direction of the human body as the positive direction of an X axis and the key node HIP CENTER as the origin;
step 3, respectively obtaining attitude labels at each moment by using the position data of the key nodes under the morphological coordinate system at each moment, which are obtained in the step 2, wherein the attitude labels comprise main body attitude labels GLbodyLeft forelimb posture label GL1fRight front limb posture label GLrfLeft hind limb posture label GL1bAnd right hind limb posture label GLrb；
The subject posture label GL in the step 3bodyThe method for obtaining (1) is as follows:
selecting XF，YFAnd ZFFinding the coordinate value with the maximum absolute value, and finding the GL corresponding to the interval to which the coordinate value belongsbodyIs the subject posture label GLbodyThe following formula is adopted:
Figure FDA0002742446600000021
wherein, XF，YFAnd ZFCoordinates of 3 coordinate axes of the unit vector F respectively; unit vector
Figure FDA0002742446600000022
Figure FDA0002742446600000023
Figure FDA0002742446600000024
A vector formed by the key node HEAD and the key node HIP CENTER in the skeleton tracking device coordinate system;
the left forelimb posture label GL in the step 31fRight front limb posture label GLrfLeft hind limb posture label GL1bAnd right hind limb posture label GLrbThe method for obtaining (1) is as follows:
the four posture labels all comprise three key nodes which are marked as a key node 1, a key node 2 and a key node 3, and the posture label GL of the left forelimb1fThree key nodes included are ELBOW LEFT, WRIST LEFT and HAND LEFT, respectively, for the right forelimb posture label GLrfThree key nodes included are KNIEE LEFT, ANIKLE LEFT and FOOT LEFT, respectively, for LEFT hind limb posture label GL1bThe three key nodes are ELBOW LEFT, WRIST LEFT and HAND LEFT, right hind limb posture label GLrbThe three key nodes are KNIEE LEFT, ANIKLE LEFT and FOOT LEFT respectively;
the data of the three key nodes in the morphological coordinate system are respectively used as (X)1，Y1，Z1)(X2，Y2，Z2)(X3，Y3，Z3) Represents; the four attitude tags each include height tag G1, orientation tag G2, and curl tag G3;
the height label G1 is obtained by the following method:
G1＝(g1+g2+g3) And/3, rounding, wherein,
Figure FDA0002742446600000031
wherein n is 1,2,3, YHIs a Y-axis coordinate in the morphological coordinate system of the key node HEADHCThe Y-axis coordinate of the key node SHOULDER CENTER under the morphological coordinate system;
the orientation label G2 is obtained as follows:
counting the symbols of the X-axis coordinate and the Z-axis coordinate of the key node 1, the key node 2 and the key node 3, and solving an orientation label G2 by adopting the following formula:
Figure FDA0002742446600000032
the method for obtaining the crimp label G3 is as follows:
introducing a key node 4 according to the key node 1, the key node 2 and the key node 3, respectively calculating the distances D between the key node 1, the key node 2 and the key node 3 and the key node 41，D2，D3(ii) a For left forelimb posture label GL1fThe key node 4 is SHOULDER LEFT, for the right forelimb posture label GLrfThe key node 4 is SHOULDER RIGHT, for left hind limb posture label GL1bThe key node 4 is HIPLEFT, and the right hind limb posture label GLrbThe key node 4 is HIPRIGHT;
the value of the curl label G3 is given by the following formula:
Figure FDA0002742446600000033
2. a method for obtaining an action template library, comprising the steps of:
step 1, making standard actions for multiple times, and decomposing the standard actions made each time into attitude tags at each moment; selecting the attitude tag at the initial moment as an initial frame attitude tag, and selecting the attitude tag at the termination moment as a termination frame attitude tag; taking the standard action made for the first time as a comparison standard action, and taking the standard actions made for other times as reference standard actions; comparing the initial frame posture label corresponding to the standard action as an initial frame comparison posture label, and using the termination frame posture label corresponding to the first made standard action as a termination frame comparison posture label;
the standard action made each time is decomposed into attitude tags at various moments, wherein the attitude tags are obtained according to the method of claim 1;
step 2, solving an initial frame similarity coefficient group, wherein the specific method is as follows:
respectively calculating the similarity sl1(A) of each attribute of the start frame attitude tag and the start frame contrast attitude tags of a plurality of reference standard actionsnThe formula used is as follows:
Sl1(A)n＝An×z1n÷ln(n∈Z，n∈[1，13]when n is 1,4,7,10,13, ln is 5, and the rest is 3)
Wherein A isnFor initialized similarity coefficient values, AnN denotes the serial number of the attribute, and the serial numbers 1 to 13 are the body posture labels GL of the posture labels, respectivelybodyLeft forelimb posture label GL1fHeight label G1 in (1), left forelimb posture label GL1fMiddle orientation label G2, left forelimb posture label GL1fMiddle curl tag G3, left hind limb posture tag GL1bHeight label G1 in, left hind limb posture label GL1bMiddle orientation label G2, left hind limb posture label GL1bMiddle curl label G3, right front limb posture label GLrfHeight label G1, right front limb posture label GLrfOrientation label G2, right forelimb posture label GLrfThe curl label G3, the right hind limb posture label GLrbHeight label G1, right hind limb posture label GLrbOrientation label G2, right hind limb posture label GLrbCrimp label G3; z1nThe absolute value of the difference value of the corresponding attributes of the initial frame attitude tag and the initial frame contrast attitude tag of the reference standard action;
for each attribute n, selecting similarity Sl1(A) calculated according to the starting frame attitude labels of a plurality of reference standard actionsnThe second largest value in (b) is taken as the similarity coefficient value A1 under the attributenThe similarity coefficient value A1 corresponding to each attribute nnForming an initial frame similarity coefficient set Astar＝{A1n，n∈Z，n＝1，2，...，13}；
And step 3, solving a similarity coefficient group of the termination frame, wherein the specific method comprises the following steps:
respectively calculating the similarity Sl2(A) of each attribute of the terminal frame attitude tag and the terminal frame contrast attitude tags of a plurality of reference standard actionsnThe formula used is as follows:
Sl2(A)n＝An×z2n÷ln(n∈Z，n∈[1，13]when n is 1,4,7,10,13, ln is 5, and the rest is 3)
Wherein, z2nThe absolute value of the difference between the corresponding attributes of the terminating frame pose tag and the terminating frame contrast pose tag for the reference standard action;
selecting similarity Sl2(A) calculated according to the attitude labels of the termination frames of a plurality of reference standard actions for each attribute nnThe second largest value in (b) is taken as the similarity coefficient value A2 under the attributen(ii) a The similarity coefficient value A2 corresponding to each attribute nnForming a group A of termination frame similarity coefficientsstop＝{A2n，n∈Z，n＝1，2，...，13}；
And 4, aiming at a plurality of standard actions, obtaining an initial frame similarity coefficient group and an ending frame similarity coefficient group corresponding to each standard action according to the method in the step 1-3, wherein the initial frame similarity coefficient group and the ending frame similarity coefficient group corresponding to all the standard actions form an action template library.
3. A motion recognition method based on attitude tags is characterized by comprising the following steps:
step 1, decomposing the action to be recognized into attitude tags at each moment aiming at the action to be recognized; the gesture label for decomposing the action to be recognized into each moment is obtained according to the method of claim 1;
step 2, selecting a certain standard action in the action template library, calculating the similarity SL (B) n of each attribute between the termination frame attitude tag obtained in the step 1 and the termination frame attitude tag of the selected standard action, and recording the termination frame attitude tag as a tth frame attitude tag, wherein the adopted formula is as follows:
SL(B)n＝A1n×z3n÷ln(n∈Z，n∈[1，13]when n is 1,4,7,10,13, ln is 5, and the rest is 3)
Wherein, z3nSelecting similarity Sl1(A) calculated for a plurality of initial frame attitude tags of reference standard actions for the absolute value of the difference between the attributes corresponding to the end frame attitude tag obtained in step 1 and the end frame attitude tag of the selected standard actionnThe second largest value in (b) is taken as the similarity coefficient value A1 under the attributen；
Calculating the overall similarity S (B) between the attitude tag of the termination frame obtained in the step 1 and the attitude tag of the termination frame of the selected standard action, wherein the formula is as follows:
Figure FDA0002742446600000061
step 3, if the overall similarity S (B) is greater than the set threshold MAXBLUR, returning to the step 2; otherwise, executing step 4;
step 4, calculating the similarity SL (C) of each attribute between the previous frame attitude tag of the ending frame attitude tag and the starting frame attitude tag of the selected standard actionnAnd recording a frame attitude tag as a t-1 frame attitude tag, wherein the adopted formula is as follows:
SL(C)n＝A2n×z4n÷ln(n∈Z，n∈[1，13]when n is 1,4,7,10,13, ln is 5, and the rest is 3)
Wherein, z4nSelecting similarity Sl2(A) calculated for a plurality of end frame attitude tags of reference standard actions for the absolute value of the difference between the corresponding attributes of the previous frame attitude tag and the start frame attitude tag of the selected standard actionnThe second largest value in (b) is taken as the similarity coefficient value A2 under the attributen；
Calculating the overall similarity S (C) between the pose label of the last frame and the pose label of the initial frame of the selected standard action by the following formula:
Figure FDA0002742446600000062
step 5, if the overall similarity S (C) is less than a set threshold MAXBLUR, the action to be recognized is consistent with the selected standard action; and if the overall similarity S (C) is greater than the set threshold MAXBLUR, returning to the step 4, replacing the posture label of the t-1 frame of the processed object with the posture label of the t-2 frame of the processed object until the overall similarity S (C) is greater than the set threshold MAXBLUR when the processed object is the posture label of the first frame, and returning to the step 2.
CN201810133363.0A 2018-02-09 2018-02-09 Gesture tag-based action recognition method Expired - Fee Related CN108392207B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810133363.0A CN108392207B (en) 2018-02-09 2018-02-09 Gesture tag-based action recognition method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810133363.0A CN108392207B (en) 2018-02-09 2018-02-09 Gesture tag-based action recognition method

Publications (2)

Publication Number Publication Date
CN108392207A CN108392207A (en) 2018-08-14
CN108392207B true CN108392207B (en) 2020-12-11

Family

ID=63096010

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810133363.0A Expired - Fee Related CN108392207B (en) 2018-02-09 2018-02-09 Gesture tag-based action recognition method

Country Status (1)

Country Link
CN (1) CN108392207B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110215216B (en) * 2019-06-11 2020-08-25 中国科学院自动化研究所 Behavior identification method and system based on skeletal joint point regional and hierarchical level
CN110309743A (en) * 2019-06-21 2019-10-08 新疆铁道职业技术学院 Human body attitude judgment method and device based on professional standard movement
CN112617819A (en) * 2020-12-21 2021-04-09 西南交通大学 Method and system for recognizing lower limb posture of infant

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11272877A (en) * 1998-03-25 1999-10-08 Namco Ltd Skeleton model data representation
CN104268138A (en) * 2014-05-15 2015-01-07 西安工业大学 Method for capturing human motion by aid of fused depth images and three-dimensional models
CN105608467A (en) * 2015-12-16 2016-05-25 西北工业大学 Kinect-based non-contact type student physical fitness evaluation method
CN106022213A (en) * 2016-05-04 2016-10-12 北方工业大学 Human body motion recognition method based on three-dimensional bone information
KR101722131B1 (en) * 2015-11-25 2017-03-31 국민대학교 산학협력단 Posture and Space Recognition System of a Human Body Using Multimodal Sensors
CN107115102A (en) * 2017-06-07 2017-09-01 西南科技大学 A kind of osteoarticular function appraisal procedure and device
CN107174255A (en) * 2017-06-15 2017-09-19 西安交通大学 Three-dimensional gait information gathering and analysis method based on Kinect somatosensory technology
CN107225573A (en) * 2017-07-05 2017-10-03 上海未来伙伴机器人有限公司 The method of controlling operation and device of robot

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6334925B2 (en) * 2013-01-18 2018-05-30 キヤノンメディカルシステムズ株式会社 Motion information processing apparatus and method
CN103886588B (en) * 2014-02-26 2016-08-17 浙江大学 A kind of feature extracting method of 3 D human body attitude projection
WO2015162158A1 (en) * 2014-04-22 2015-10-29 Université Libre de Bruxelles Human motion tracking
CN105243375B (en) * 2015-11-02 2018-05-18 北京科技大学 A kind of motion characteristic extracting method and device
CN106528586A (en) * 2016-05-13 2017-03-22 上海理工大学 Human behavior video identification method
CN106295616B (en) * 2016-08-24 2019-04-30 张斌 Exercise data analyses and comparison method and device
CN106874884B (en) * 2017-03-03 2019-11-12 中国民航大学 Human body recognition methods again based on position segmentation
CN107038430B (en) * 2017-05-05 2020-09-11 成都通甲优博科技有限责任公司 Method and device for constructing human body posture data sample

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11272877A (en) * 1998-03-25 1999-10-08 Namco Ltd Skeleton model data representation
CN104268138A (en) * 2014-05-15 2015-01-07 西安工业大学 Method for capturing human motion by aid of fused depth images and three-dimensional models
KR101722131B1 (en) * 2015-11-25 2017-03-31 국민대학교 산학협력단 Posture and Space Recognition System of a Human Body Using Multimodal Sensors
CN105608467A (en) * 2015-12-16 2016-05-25 西北工业大学 Kinect-based non-contact type student physical fitness evaluation method
CN106022213A (en) * 2016-05-04 2016-10-12 北方工业大学 Human body motion recognition method based on three-dimensional bone information
CN107115102A (en) * 2017-06-07 2017-09-01 西南科技大学 A kind of osteoarticular function appraisal procedure and device
CN107174255A (en) * 2017-06-15 2017-09-19 西安交通大学 Three-dimensional gait information gathering and analysis method based on Kinect somatosensory technology
CN107225573A (en) * 2017-07-05 2017-10-03 上海未来伙伴机器人有限公司 The method of controlling operation and device of robot

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Motion Capture of the Human Body Using Multiple Depth Sensors;Kim, Yejin; Baek, Seongmin; Bae, Byung-Chull;《Etri Journal》;20170430;第39卷(第2期);全文 *
SkeletonNet: Mining Deep Part Features for 3-D Action Recognition;Ke Q , An S , Bennamoun M , et al.;《IEEE Signal Processing Letters》;20170630;第24卷(第6期);全文 *

Also Published As

Publication number Publication date
CN108392207A (en) 2018-08-14

Similar Documents

Publication Publication Date Title
CN109086706B (en) Motion recognition method based on segmentation human body model applied to human-computer cooperation
US10719759B2 (en) System for building a map and subsequent localization
CN107103613B (en) A kind of three-dimension gesture Attitude estimation method
CN109597485B (en) Gesture interaction system based on double-fingered-area features and working method thereof
Bhuyan et al. Fingertip detection for hand pose recognition
Feng et al. Real-time fingertip tracking and detection using Kinect depth sensor for a new writing-in-the air system
CN102567703B (en) Hand motion identification information processing method based on classification characteristic
CN107368820B (en) Refined gesture recognition method, device and equipment
CN108392207B (en) Gesture tag-based action recognition method
CN105868716A (en) Method for human face recognition based on face geometrical features
Yao et al. Real-time hand pose estimation from RGB-D sensor
CN104821010A (en) Binocular-vision-based real-time extraction method and system for three-dimensional hand information
CN111857334A (en) Human body gesture letter recognition method and device, computer equipment and storage medium
Uddin et al. Human Activity Recognition via 3-D joint angle features and Hidden Markov models
CN108491754A (en) A kind of dynamic representation based on skeleton character and matched Human bodys' response method
CN111444488A (en) Identity authentication method based on dynamic gesture
CN109543644A (en) A kind of recognition methods of multi-modal gesture
CN112800990B (en) Real-time human body action recognition and counting method
JP2016014954A (en) Method for detecting finger shape, program thereof, storage medium of program thereof, and system for detecting finger shape
Kumar et al. 3D sign language recognition using spatio temporal graph kernels
CN107220634B (en) Based on the gesture identification method for improving D-P algorithm and multi-template matching
CN114495273A (en) Robot gesture teleoperation method and related device
Jiang et al. independent hand gesture recognition with Kinect
Yashas et al. Hand gesture recognition: a survey
Hassanpour et al. Visionbased hand gesture recognition for human computer interaction: A review

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20201211

Termination date: 20220209