CN107679512A - A kind of dynamic gesture identification method based on gesture key point - Google Patents
A kind of dynamic gesture identification method based on gesture key point Download PDFInfo
- Publication number
- CN107679512A CN107679512A CN201710983280.6A CN201710983280A CN107679512A CN 107679512 A CN107679512 A CN 107679512A CN 201710983280 A CN201710983280 A CN 201710983280A CN 107679512 A CN107679512 A CN 107679512A
- Authority
- CN
- China
- Prior art keywords
- point
- gesture
- calculating
- dynamic
- finger
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a kind of dynamic gesture identification method based on gesture key point, belong to computer gesture identification field.It is characterized in that：Methods described determines finger fingertip and refers to location of root by obtaining the profile point coordinates of human hand counterclockwise, and conventional dynamic gesture is identified according to the direction of motion, distance and the angle of finger tip and finger root；The dynamic gesture includes：Crawl, discharge, translation, turn clockwise, rotate counterclockwise and before push away.The present invention proposes the dynamic hand gesture recognition algorithm of a kind of position based on finger tip, distance and direction.It can avoid gesture and the identification impact of external noise of inner chamber.By many experiments, the present invention can identify grasping, place, turn clockwise, rotate counterclockwise, and four direction translation and to promote state gesture identification rate forward be 96%.
Description
Technical Field
The invention belongs to the field of computer gesture recognition, and particularly relates to a dynamic gesture recognition method based on gesture key points.
Background
The process of gesture recognition mainly comprises four stages: gesture image acquisition, gesture image preprocessing using some techniques including edge detection, filtering and normalization, extraction of gesture main features, and identification (or classification) stage. There are many ways in gesture recognition. For example, bailador uses a continuous time-cycled neural network for gesture recognition Starner and Sorrentino use a real-time HMM based system for recognizing gestures. Then a significant region such as the Maximally Stable Extremal Region (MSER) is detected on the motion divergence map. From each detected region, local descriptors are extracted to capture local motion patterns. And then further utilizes indexing techniques to search for gestures from the image. Reyes identifies gestures using weighted dynamic time programming. Dun [7] proposes a real-time gesture recognition system by using shape context matching and cost matrices. Panwar proposes a real-time system for gesture recognition based on the detection of some meaningful shape-based feature centers (centroids), finger states, thumb in lifting or folding fingers of the hand. And this simple shape method based on gesture recognition can recognize about 45 different gestures based on a 5-bit binary string as the output of the algorithm. This proposed implementation algorithm has tested 450 images, which gives an approximate recognition rate of 94%. Chen proposes to recognize gestures by detecting palms and fingers and achieves very good results. Rucklidge recognizes gestures using the Hausdorff distance, his method is widely used. For example, yang scholan, uses Hausdorff to achieve good results in static gesture recognition. He also achieved good recognition results using this method in dynamic gesture recognition of a 9-frame image sequence. The biggest disadvantage of his proposed method is that he is required to move evenly, has to complete 9 frames, he cannot recognize very useful gestures, such as clockwise or counter-clockwise rotation (because of its rotational invariance), and gestures forward.
Disclosure of Invention
The invention aims to solve the problems in the prior art and provides a dynamic gesture recognition method based on a gesture key point, which is used for performing dynamic gesture recognition based on the position, distance and direction of a fingertip.
The invention is realized by the following technical scheme:
a dynamic gesture recognition method based on gesture key points comprises the steps of obtaining contour point coordinates of a human hand anticlockwise, further determining positions of finger tips and finger roots, and recognizing common dynamic gestures according to movement directions, distances and angles of the finger tips and the finger roots; the dynamic gesture includes: grasping, releasing, translating, rotating clockwise, rotating counterclockwise, and pushing forward.
The method comprises the following steps:
firstly, acquiring a frame of image from a common camera;
secondly, segmenting the image by using a skin color model method and carrying out binarization on the image, wherein if pixel points at the ith row and j column are skin color points, f (i, j) =1;
thirdly, calculating the centroid coordinate of the binarized image, obtaining the coordinate of a point on the gesture contour anticlockwise, and recording the length C of the contour;
fourthly, calculating the number of fingers, and recording the positions of the found fingertips and roots as P 1i And P 2i Calculating the distance between adjacent fingertips as H 1 (i, i + 1), calculating the distance between adjacent finger roots and recording as H 2 (i, i + 1), calculating the distance between the finger root and the adjacent fingertip and recording as H 12 (i,i+1),H 21 (i, i) recording coordinates of the middle finger tip and two roots thereof, and calculating a counterclockwise angle between a horizontal axis and a line connecting the middle finger center and the centroid coordinates;
and fifthly, recognizing the dynamic gesture according to the binary decision tree.
The third step of calculating the coordinates of the centroid of the binarized image is implemented as follows:
calculating the centroid coordinates according to a centroid coordinate formula:
wherein xo is the abscissa of the centroid point and yo is the ordinate of the centroid point; f (i, j) is a value at a coordinate point (i, j), wherein i represents a row and j represents a column, and if a pixel point at the ith row and the jth column is a skin color point, f (i, j) =1.
The coordinate of the point on the gesture outline is obtained counterclockwise in the third step by:
step 1: determining the position of the starting point: searching skin colors line by line from two lines at the lower left of the gesture bounding box, recording the number N of skin color points, and marking the first skin color point on the upper line as a starting point P when N is more than 10 start Copying the coordinates of the starting point to the point X;
and 2, step: determining an initial direction: searching 8 neighborhood pixels of a current point X, if the lower right corner is a background point, searching 8 neighborhoods from the position of 0 in a counterclockwise way until the position of a first skin color point is searched, and assigning the position to d; if the lower right side is a flesh tone point, searching 8 neighborhoods from the position of 0 anticlockwise, recording the last position of a plurality of continuous background points in the 8 neighborhoods, copying the next position of the position to d, and if the background points are divided into two equal parts, assigning the next position of the position where the first background point is located to d, thereby obtaining an initial direction d;
and step 3: determine the location of the next point: copying the coordinates at d to the current point X, and starting to search the 8 neighborhoods of X from the position of d-2 until searching the first flesh tone point P, and recording the position d' of P;
and 4, step 4: d' is copied to d, and then step 3 is repeated until the motion point P and the starting point P start Overlapping;
and 5: counting the number N of edge points, judging whether the number N of times is more than 70, and if so, ending the search; if it is less, the search line will be moved up by 5 lines and a count N is assigned to 0, and the first step is restarted. The search ends until N is greater than 70.
The fifth step is realized by:
condition C1: judging whether the number of the fingers is suddenly changed, if so, if the number is reduced, indicating that the finger is released in a sudden change state S3=1, and returning to the first step; s3=2 if the number becomes large, indicating seizing, and returning to the first step; if the number is not changed, judging the right branch;
condition C2: judging the number of the current fingers, if the number is equal to 5, entering a left branch, and otherwise, entering a right branch;
condition C3: judging whether to advance or not, if the coordinate of the mass center does not change greatly and the contour length changes suddenly, indicating that the vehicle advances in a state of S0=1, and returning to the first step; otherwise, judging the right branch;
condition C4: judging whether the person is translated or not, if the change is large, calculating the moving direction of the hand, wherein the state identifier S1=1 of the motion of the translation gesture represents upward, S1=2 represents downward, S1=3 represents leftward, S1=4 represents rightward, and then returning to the first step;
judging whether the hand moves horizontally or not, and if the H1i, H2i and H21 are basically the same as the mass center change range, calculating the moving direction of the hand;
condition C5: and determining whether to rotate according to the trend of the angle change, if the angle is increased, rotating the mark S2=1 of anticlockwise rotation clockwise to indicate clockwise rotation, and if the angle is decreased, rotating the mark S2=2 to indicate anticlockwise rotation, and then returning to the first step.
Compared with the prior art, the invention has the beneficial effects that: the invention provides a dynamic gesture recognition algorithm based on positions, distances and directions of fingertips. It can avoid the gesture of inner chamber and the discernment of external noise to strike. Through a large number of experiments, the gesture recognition rate of the four-direction translation and forward pushing state of the hand-held gesture recognition device can be 96% in the aspects of grasping, placing, clockwise rotating, anticlockwise rotating and translation and forward pushing in four directions.
Drawings
FIG. 1 is a schematic diagram of the determination of initial position in the method of the present invention
FIG. 2 is a block diagram of the steps of the method of the present invention
FIG. 3 is a binary decision tree
FIG. 4 is a graph comparing experimental results.
Detailed Description
The invention is described in further detail below with reference to the accompanying drawings:
image Segmentation methods can be divided into two categories, one is motion-Based Gesture Segmentation (including background Difference Method (see "Yuan Min, yao Heng, liu Jian. Dynamic Collection Combining Three-frame Difference Method and Skin-color Electronic creation Model [ J ]. Opto-Electronic Engineering,2016, 06), optical Flow Segmentation Method (see" Xu Yanqun, zhang Bin. Application of Segmentation Based on Optical Flow in light registration [ J ]. Computer Science,2012,04, 275-277 292 ") and the other is Based on Skin color threshold value. Common threshold segmentation methods include: HSV threshold methods (conversion formulae (see "Tan Wenjun. Research on Algorithm and Model of Hand Gestures registration Based on Computer Vision [ D ]. Northeastern University, 2010"): formula 1) and YcbCr threshold methods (conversion formulae (see "Tan Wenjun. Research on Algorithm and Model of Hand Gestures registration Based on Computer Vision [ D ]. Northtern University, 2010"): formula 2). The two methods are suitable for skin color clustering, but the RGB-YCbCr conversion method is relatively simple compared with the other conversion method, and the YcbCr spatial threshold method is used for segmentation.
Acquiring contour points anticlockwise: an edge of an image refers to a set of pixels having a large change in gray level, and a gradient is typically used to represent a change in gray level of the image. The gradient vector of image f (x, y) at point (x, y) is ^ f, defined as follows:
the common edge detection operators are mainly: sobel Operator (refer to "Zhou Xuehai, zhang wu.. Multiscale Edge Detection Based on the Sobel Operator [ J ]. Microelectronics and Computer Science,2006, 12-14+ 18"), prewitt Operator (refer to "Li Haiyang, wen yongge.color Image Detection Based on Prewitt [ J ]. FuJian Computer,2013, 05): yang Shanlin, gray Image's Edge Detection Based on Gauss-Laplace [ J ]. Computer Engineering and Applications,2003, 26.
Further, while a one-and-one approach (refer to "Xiangkun tseng. Dynamic gesture recognition of hand shape feature combined with motion trajectory [ D ]. Shanghai landmark University, 2005.") proposes eight neighborhood coding methods for extracting contours, he describes a procedure that does not fully extract edge point coordinates. The invention modifies the algorithm as follows: (the algorithm is part of the feature point extraction in FIG. 2. The feature point extraction in FIG. 2 includes obtaining the coordinates of the contour points and finding the discontinuities (extracting the discontinuities from the obtained contour point coordinates), determining the position of the finger root and the finger tip (determining the position coordinates of the finger root and the finger tip in the discontinuities)
Step 1: the starting point position is determined. Starting from the two lines at the bottom left of the gesture bounding box, the skin color is searched line by line, and the number N of skin color dots is recorded. When N is greater than 10, mark the first flesh tone dot on the previous row as the starting point P start . Copying the coordinates of the starting point to the point X;
the method comprises the following steps: traversing each point in the binary image from left to right, and solving the minimum value of coordinates of the skin color point; traversing each point in the binary image from right to left to obtain the maximum value of the coordinates of the skin color point; traversing each point in the binary image from top to bottom, and solving the minimum value of the coordinates of the skin color point; and traversing each point in the binary image from bottom to top, and solving the maximum value of the coordinates of the skin color points to obtain the gesture bounding box.
And 2, step: an initial direction is determined. Searching 8 neighborhood pixels of a current point X, if the lower right corner is a background point, searching 8 neighborhoods from the position of 0 counterclockwise until the position of a first skin color point is searched, and assigning the position to d; if the skin color point is on the lower right, searching 8 neighborhoods from the position of '0' counterclockwise, recording the last position of a plurality of continuous background points in the 8 neighborhoods, copying the next position of the position to d, and if the background points are divided into two parts which are equal, assigning the next position of the position where the first background point is located to d, thereby obtaining the initial direction d. The search graph is shown in fig. 1.
And step 3: the position of the next point is determined. The coordinates at d are copied to the current point X and an 8-neighborhood of X is searched starting from the position of d-2 until the first flesh tone point P is searched. Record position d 'of P'
And 4, step 4: copy d' to d. Repeating the step 3 until the motion point P and the starting point P start Coincidence
And 5: counting the number N of edge points, judging whether the number N of times is more than 70, and if so, ending the search; if it is less, the search line will be moved up by 5 lines and the count N will be assigned to 0, and the first step will be restarted. The search is ended until N is greater than 70.
The invention provides a novel dynamic gesture recognition algorithm. The contour point coordinates of a human hand are acquired anticlockwise, so that the positions of finger tips and finger roots are determined, and common dynamic gestures are identified according to the movement directions, distances and angles of the finger tips and the finger roots, and the method comprises the following steps: grasping, releasing, translating, rotating clockwise, rotating counterclockwise, pushing forward, etc. The method is basically not influenced by the speed of the movement of the human hand, and can well solve the problems caused by poor image segmentation, such as: the influence of a hole inside the gesture and the interference of noise outside the gesture. A large number of experiments show that the algorithm provided by the invention can well identify dynamic gestures such as grabbing and releasing, translation, clockwise rotation, anticlockwise rotation and the like, and the identification rate reaches 96%.
Extracting feature points
1) Finding mutation points
Step 1: starting from the first point on the stored contour, three consecutive points of different horizontal and vertical coordinates are sought. Recorded as a fixed point P (x 0, y 0).
Step 2: moving forward 2 points from the fixed point, the moved point P1 (x 1, y 1) among the stored adjacent contour points is obtained.
And step 3: the distance between P1 and P is calculated according to equation 4 and recorded as H i
And 4, step 4: moving forward from P1 by 2 points, denoted as P2, calculating the distance between P2 and P according to equation 4, denoted as H2
And 5: comparing H1 and H2, if H2> H1, then H2 is copied to H1, returning to step 4; if H2 is less than H1, recording the frequency n and judging whether n is more than 6, if n is less than 6, returning to the step 4; if n >6, the coordinates A1 of 12 points after this point (this point is the fingertip or the base of the finger) are recorded. Copy A1 to P, return to step 2. Until all contours are found
2) Determining finger tip and root
Among the obtained points, the distance Hi between the spaced points and the distance Di between the adjacent points are calculated according to equation 5
If Hi < Di and Hi < Di +1, then this point is a finger root.
Finding two or three root markers is marked with green dots, and other dots are marked with blue or no marks. The coordinates of the fingertips and roots are now available.
And (3) dynamic gesture recognition:
the dynamic gesture mainly comprises: grabbing action and releasing action; horizontal translation (eight orientations); horizontally rotating; push forward, etc.
The flow chart is shown in fig. 2, and comprises:
firstly, acquiring a frame of image with the width of 400 and the height of 300 from a common camera; 400 and 300 are not necessarily required to be used with the present invention other values, such as 500, 500or 600,800;
and secondly, segmenting the image by using a skin color model method and binarizing the image, wherein if the pixel point at the ith row and j column is a skin color point, f (i, j) =1.
Thirdly, calculating the coordinates of the mass center of the image after binaryzation,
according to the formula of mass center coordinatesWhere xo is the abscissa of the centroid point and yo is the ordinate of the centroid point. f (i, j) is a value at coordinate point (i, j), where i represents a row and j represents a column. If the pixel point at the ith row and the j column is the skin color point f (i, j) =1
Coordinates of points on the gesture outline are obtained counterclockwise and the length C of the outline is recorded.
Fourthly, calculating the number of fingers, and recording the positions of the found fingertips and roots as P 1i And P 2i Calculating the distance between adjacent fingertips as H 1 (i, i + 1), calculating the distance between adjacent finger roots and recording as H 2 (i, i + 1), calculating the distance between the finger root and the adjacent fingertip and recording as H 12 (i,i+1),H 21 (i, i) recording the coordinates of the middle finger tip and its two roots, and calculating the counterclockwise angle between the horizontal axis and the line connecting the center of the middle finger and the coordinates of the center of mass according to formula 6
If(||m 1 *n 1 -1||＜0.0001)angle＝0；
Else If(||m 1 *n 1 +1||＜0.0001)angle＝180；
Else If(x 1 *y 2 -x 2 *y 1 ＜0)
angle =360-acos (m 1, n 1) (whether 1, 2 is a subscript or not)
Else
angle＝acos(m1,n1)
m1 and n1 are two vectors, m1 represents the negative direction of the x axis in the plane xoy, and n1 represents a unit vector of the direction in which the connection line of the center and the centroid coordinate is located; x1 and y1 represent x coordinates and y coordinates of a vector connecting the center of the middle finger and the coordinate of the center of mass; angle represents the angle between the two vectors; acos inverse cosine function in mathematics
In a fifth step, the dynamic gesture is recognized based on a binary decision tree (as shown in FIG. 3)
Condition C1: judging whether the number of the fingers is suddenly changed, if so, reducing the number, and S3 (whether the number of the fingers is suddenly changed, namely the sudden change is not 0, and the finger is not suddenly changed to 0) =1 to release and returning; s3=2 indicates seizing and returning to the first step if the number becomes large; if the number is not changed, the right branch is determined.
Condition C2: judging the number of the current fingers, if the number is equal to 5, entering a left branch, and if not, entering a right branch;
condition C3: judging whether the experimenter pushes forwards or not, if the coordinate of the mass center does not change greatly and the length of the outline changes suddenly, S0 (the state is equal to 1 to push forwards) =1 to push forwards, and returning; otherwise, judging the right branch.
Condition C4: judging whether the person is translating or not, if the change is large, calculating the moving direction of the hand according to formula X (more precisely, the code under C4'), S1 (status indication of the translation gesture movement, the de-differentiated value indicates a different direction) =1 indicates upward, S1=2 indicates downward, S1=3 indicates leftward, S1=4 indicates rightward, and then returning to
And in the condition C4', judging whether the human hand translates, and if the H1i, the H2i and the H21 are basically the same as the mass center change range S1, translating, calculating the moving direction of the human hand according to the following formula X (code).
If(||x 1 -x 2 ||＞||y 1 -y 2 ||)
If(y 1 >y 2 )S1＝1
Else S1＝2
Else
If(x 1 >x 2 )S1＝4
Else S1＝3
Wherein (x 1, y 1) is the current point and (x 2, y 2) is the previous point
Condition C5: whether to rotate is determined according to the trend of the angle change, if the angle becomes larger S2 (flag of clockwise rotation and counterclockwise rotation) =1 represents that: clockwise rotation, if the angle becomes smaller by S2=2, it means counterclockwise rotation. Or return to the first step.
The experiment is compared with the dynamic gesture recognition provided by the Yang scholar text, the same segmentation method is adopted, the contour is extracted, and key points are searched for dynamic gesture recognition; the Yankee language maps the divided hands into a 32X32 region, then establishes a coordinate system according to a Main Direction of a Gesture (refer to "Yang Xuewen, feng Zhiquan, huang Zhongzhu, he Na. Gesture registration Based on Combining Main orientation of Gesture and Hausdorff-like Distance [ J ]. Journal of Computer-aid Design & Computer Graphics,2016,01: 307-311') (see" Liu H, feng S, zha H, et al. Document Image Retrieval Based on sensitivity Distribution Feature and Key Block Feature [ C ]// origin International Conference on assessment and registration.2005: 1040-1044 "), and then identify dynamic gestures using the class-Hausdorff Distance.
And (3) acquiring experimental data, namely, under the ordinary laboratory environment of 50 volunteers, adjusting the position of a camera to ensure that the face does not appear in an image acquisition area, and ensuring that the palm is over against the camera (can rotate properly), so that the volunteers naturally complete the following 9 dynamic gestures 20 times respectively. Comprises grabbing, placing, moving upwards, moving downwards, moving leftwards, moving rightwards, rotating clockwise, rotating anticlockwise and pushing forwards. The gesture collection process does not specify speed, sequence, etc. And finally, screening out 500 dynamic sequences of the 9 dynamic gestures without the motion blur pictures.
Dynamic gesture recognition is performed on the data set by using the dynamic gesture recognition method of the Yang academic text and the dynamic recognition method provided by the text, and the experimental result is imported into MATLAB to obtain a comparison graph of the two methods, as shown in FIG. 4. Experiments show that the dynamic gesture recognition provided by the Yang scholar language cannot recognize clockwise rotation, anticlockwise rotation, forward pushing and other dynamic gestures, the capturing and releasing recognition effect is poor, the translation gesture recognition effect is common, and the recognition effect of the method provided by the text is more than 96%.
Although the haustoroff-like distance used by the Yanhusband is among static gestures, in many static gestures, recognition works well. But since it has rotation invariance and zoom invariance, it cannot recognize the push-forward as well as the rotation gesture. In the dynamic gesture with an unfixed frame number, the algorithm cannot accurately find the key frame for feature matching, so that the recognition effect of the method in the dynamic gesture of translation and grabbing and releasing is poor. In comparison, the algorithm in the text recognizes based on the positions of the finger tips and the finger roots of the gesture, eliminates the influences of the speed of the gesture and the holes in the gesture, is certainly not interfered by external miscellaneous points, and has a good recognition effect in the grabbing and releasing translation; according to the change of the positions (clockwise included angles) of the fingertips and the finger roots relative to the mass center in the rotating process of the human hand, the trend of identifying the clockwise or counterclockwise rotation also has good effect
The above-described embodiments are intended to be illustrative only, and various modifications and variations such as those described in the above-described embodiments of the invention may be readily made by those skilled in the art based upon the teachings and teachings of the present invention without departing from the spirit and scope of the invention.
Claims (5)
1. A dynamic gesture recognition method based on gesture key points is characterized in that: according to the method, the coordinates of contour points of a human hand are acquired anticlockwise, so that the positions of fingertips and finger roots are determined, and common dynamic gestures are recognized according to the movement directions, distances and angles of the fingertips and the finger roots; the dynamic gesture includes: grabbing, releasing, translating, rotating clockwise, rotating counterclockwise and pushing forward.
2. The method of claim 1 for dynamic gesture recognition based on gesture keypoints, characterized in that: the method comprises the following steps:
firstly, acquiring a frame of image from a common camera;
secondly, segmenting the image by using a skin color model method and carrying out binarization on the image, wherein if pixel points at the ith row and j column are skin color points, f (i, j) =1;
thirdly, calculating the centroid coordinate of the binarized image, obtaining the coordinate of a point on the gesture contour anticlockwise, and recording the length C of the contour;
fourthly, calculating the number of fingers, and recording the positions of the found fingertips and roots as P 1i And P 2i Calculating the distance between adjacent fingertips as H 1 (i, i + 1), calculating the distance between adjacent finger roots and recording as H 2 (i, i + 1), calculating the distance between the finger root and the adjacent fingertip and recording as H 12 (i,i+1),H 21 (i, i) recording coordinates of the middle finger tip and two roots thereof, and calculating a counterclockwise angle between a horizontal axis and a line connecting the middle finger center and the centroid coordinates;
and fifthly, identifying the dynamic gesture according to the binary decision tree.
3. The method of claim 2, wherein the method comprises: the third step of calculating the coordinates of the centroid of the binarized image is realized by:
calculating the centroid coordinates according to a centroid coordinate formula:
wherein xo is the abscissa of the centroid point and yo is the ordinate of the centroid point; f (i, j) is a value at a coordinate point (i, j), wherein i represents a row and j represents a column, and if a pixel point at the ith row and the jth column is a skin color point, f (i, j) =1.
4. The method of claim 3 for dynamic gesture recognition based on gesture keypoints, characterized in that: the coordinate of the point on the gesture outline is obtained counterclockwise in the third step by:
step 1: determining a starting point position: searching skin colors line by line starting from two lower left lines of the gesture bounding box, recording the number N of skin color points, and marking the first skin color point on the upper line as a starting point P when N is more than 10 start Copying the coordinates of the starting point to the point X;
step 2: determining an initial direction: searching 8 neighborhood pixels of a current point X, if the lower right corner is a background point, searching 8 neighborhoods from the position of 0 in a counterclockwise way until the position of a first skin color point is searched, and assigning the position to d; if the lower right side is a flesh tone point, searching 8 neighborhoods from the position of 0 anticlockwise, recording the last position of a plurality of continuous background points in the 8 neighborhoods, copying the next position of the position to d, and if the background points are divided into two equal parts, assigning the next position of the position where the first background point is located to d, thereby obtaining an initial direction d;
and 3, step 3: determine the location of the next point: copying the coordinates at d to the current point X, and starting to search the 8 neighborhoods of X from the position of d-2 until searching the first flesh tone point P, and recording the position d' of P;
and 4, step 4: d' is copied to d, and then step 3 is repeated until the motion point P and the starting point P start Overlapping;
and 5: counting and recording the number N of the edge points, judging whether the number N of times is more than 70, and if so, ending the search; if it is less, the search line will be moved up by 5 lines and a count N is assigned to 0, and the first step is restarted. The search ends until N is greater than 70.
5. The method of claim 4 for dynamic gesture recognition based on gesture keypoints, characterized in that: the fifth step is realized by:
condition C1: judging whether the number of the fingers is suddenly changed, if the number is reduced, indicating that the finger number suddenly changed state S3=1 to release, and returning to the first step; s3=2 if the number becomes large, indicating seizing, and returning to the first step; if the number is not changed, judging the right branch;
condition C2: judging the number of the current fingers, if the number is equal to 5, entering a left branch, and otherwise, entering a right branch;
condition C3: judging whether to advance or not, if the coordinate of the mass center does not change greatly and the contour length changes suddenly, indicating that the vehicle advances in a state of S0=1, and returning to the first step; otherwise, judging the right branch;
condition C4: judging whether the person translates or not, if the change is large, calculating the moving direction of the hand, wherein the state mark S1=1 of the motion of the translation gesture represents upward, S1=2 represents downward, S1=3 represents leftward, S1=4 represents rightward, and then returning to the first step;
judging whether the hand moves horizontally or not under the condition C4', if the H1i, H2i and H21 are basically the same as the variation range of the mass center, calculating the moving direction of the hand, and returning to the first step;
condition C5: and determining whether to rotate according to the trend of the angle change, if the angle is increased, rotating the mark S2=1 which rotates anticlockwise clockwise to indicate the clockwise rotation, and if the angle is decreased, rotating S2=2 to indicate the anticlockwise rotation, and then returning to the first step.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710983280.6A CN107679512A (en) | 2017-10-20 | 2017-10-20 | A kind of dynamic gesture identification method based on gesture key point |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710983280.6A CN107679512A (en) | 2017-10-20 | 2017-10-20 | A kind of dynamic gesture identification method based on gesture key point |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107679512A true CN107679512A (en) | 2018-02-09 |
Family
ID=61141039
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710983280.6A Pending CN107679512A (en) | 2017-10-20 | 2017-10-20 | A kind of dynamic gesture identification method based on gesture key point |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107679512A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109145793A (en) * | 2018-08-09 | 2019-01-04 | 东软集团股份有限公司 | Establish method, apparatus, storage medium and the electronic equipment of gesture identification model |
| CN109190461A (en) * | 2018-07-23 | 2019-01-11 | 中南民族大学 | A kind of dynamic gesture identification method and system based on gesture key point |
| CN109697407A (en) * | 2018-11-13 | 2019-04-30 | 北京物灵智能科技有限公司 | A kind of image processing method and device |
| CN110245593A (en) * | 2019-06-03 | 2019-09-17 | 浙江理工大学 | A kind of images of gestures extraction method of key frame based on image similarity |
| CN110458059A (en) * | 2019-07-30 | 2019-11-15 | 北京科技大学 | A kind of gesture identification method based on computer vision and identification device |
| WO2020088069A1 (en) * | 2018-10-30 | 2020-05-07 | 北京达佳互联信息技术有限公司 | Hand gesture keypoints detection method and apparatus, electronic device, and storage medium |
| CN111626136A (en) * | 2020-04-29 | 2020-09-04 | 惠州华阳通用电子有限公司 | Gesture recognition method, system and equipment |
| CN111753764A (en) * | 2020-06-29 | 2020-10-09 | 济南浪潮高新科技投资发展有限公司 | Gesture recognition method of edge terminal based on attitude estimation |
| CN112114666A (en) * | 2020-08-25 | 2020-12-22 | 武汉海微科技有限公司 | Dynamic gesture recognition algorithm based on touch panel |
| CN112329646A (en) * | 2020-11-06 | 2021-02-05 | 吉林大学 | Hand gesture motion direction identification method based on mass center coordinates of hand |
| CN113269025A (en) * | 2021-04-01 | 2021-08-17 | 广州车芝电器有限公司 | Automatic alarm method and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102063618A (en) * | 2011-01-13 | 2011-05-18 | 中科芯集成电路股份有限公司 | Dynamic gesture identification method in interactive system |
| CN105929947A (en) * | 2016-04-15 | 2016-09-07 | 济南大学 | Scene situation perception based man-machine interaction method |
-
2017
- 2017-10-20 CN CN201710983280.6A patent/CN107679512A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102063618A (en) * | 2011-01-13 | 2011-05-18 | 中科芯集成电路股份有限公司 | Dynamic gesture identification method in interactive system |
| CN105929947A (en) * | 2016-04-15 | 2016-09-07 | 济南大学 | Scene situation perception based man-machine interaction method |
Non-Patent Citations (2)
| Title |
|---|
| 熊俊涛等: "基于视觉技术的手势跟踪与动作识别算法", 《计算机与现代化》 * |
| 蔡萌萌: "面向手势自然交互的人机相向互动模型与算法研究", 《HTTP://WWW.DOC88.COM/P-5071382345744.HTML》 * |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109190461A (en) * | 2018-07-23 | 2019-01-11 | 中南民族大学 | A kind of dynamic gesture identification method and system based on gesture key point |
| CN109190461B (en) * | 2018-07-23 | 2019-04-26 | 中南民族大学 | A kind of dynamic gesture identification method and system based on gesture key point |
| CN109145793A (en) * | 2018-08-09 | 2019-01-04 | 东软集团股份有限公司 | Establish method, apparatus, storage medium and the electronic equipment of gesture identification model |
| WO2020088069A1 (en) * | 2018-10-30 | 2020-05-07 | 北京达佳互联信息技术有限公司 | Hand gesture keypoints detection method and apparatus, electronic device, and storage medium |
| US11514706B2 (en) | 2018-10-30 | 2022-11-29 | Beijing Dajia Internet Information Technology Co., Ltd. | Method and device for detecting hand gesture key points |
| CN109697407A (en) * | 2018-11-13 | 2019-04-30 | 北京物灵智能科技有限公司 | A kind of image processing method and device |
| CN110245593A (en) * | 2019-06-03 | 2019-09-17 | 浙江理工大学 | A kind of images of gestures extraction method of key frame based on image similarity |
| CN110245593B (en) * | 2019-06-03 | 2021-08-03 | 浙江理工大学 | Gesture image key frame extraction method based on image similarity |
| CN110458059B (en) * | 2019-07-30 | 2022-02-08 | 北京科技大学 | Gesture recognition method and device based on computer vision |
| CN110458059A (en) * | 2019-07-30 | 2019-11-15 | 北京科技大学 | A kind of gesture identification method based on computer vision and identification device |
| CN111626136A (en) * | 2020-04-29 | 2020-09-04 | 惠州华阳通用电子有限公司 | Gesture recognition method, system and equipment |
| CN111626136B (en) * | 2020-04-29 | 2023-08-18 | 惠州华阳通用电子有限公司 | Gesture recognition method, system and equipment |
| CN111753764A (en) * | 2020-06-29 | 2020-10-09 | 济南浪潮高新科技投资发展有限公司 | Gesture recognition method of edge terminal based on attitude estimation |
| CN112114666A (en) * | 2020-08-25 | 2020-12-22 | 武汉海微科技有限公司 | Dynamic gesture recognition algorithm based on touch panel |
| CN112329646A (en) * | 2020-11-06 | 2021-02-05 | 吉林大学 | Hand gesture motion direction identification method based on mass center coordinates of hand |
| CN113269025A (en) * | 2021-04-01 | 2021-08-17 | 广州车芝电器有限公司 | Automatic alarm method and system |
| CN113269025B (en) * | 2021-04-01 | 2024-03-26 | 广州车芝电器有限公司 | Automatic alarm method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107679512A (en) | A kind of dynamic gesture identification method based on gesture key point | |
| CN107038424B (en) | Gesture recognition method | |
| Neumann et al. | Efficient scene text localization and recognition with local character refinement | |
| Yi et al. | Localizing text in scene images by boundary clustering, stroke segmentation, and string fragment classification | |
| Hasan et al. | Hand gesture modeling and recognition using geometric features: a review | |
| CN106960181B (en) | RGBD data-based pedestrian attribute identification method | |
| CN109684959B (en) | Video gesture recognition method and device based on skin color detection and deep learning | |
| Adamek et al. | Efficient contour-based shape representation and matching | |
| CN106297755B (en) | Electronic equipment and identification method for music score image identification | |
| Shah et al. | A novel biomechanics-based approach for person re-identification by generating dense color sift salience features | |
| CN105718552A (en) | Clothing freehand sketch based clothing image retrieval method | |
| CN111027544A (en) | MSER license plate positioning method and system based on visual saliency detection | |
| CN108256518A (en) | Detection method and detection device for character region | |
| Weerasekera et al. | Robust asl fingerspelling recognition using local binary patterns and geometric features | |
| Aly et al. | Arabic sign language recognition using spatio-temporal local binary patterns and support vector machine | |
| Hasan et al. | Real time fingers and palm locating using dynamic circle templates | |
| Zhang et al. | Research on the text detection and extraction from complex images | |
| Hartanto et al. | Preliminary design of static indonesian sign language recognition system | |
| Alghamdi et al. | Automated person identification framework based on fingernails and dorsal knuckle patterns | |
| CN101118544A (en) | Method for constructing picture shape contour outline descriptor | |
| CN110516638B (en) | Sign language recognition method based on track and random forest | |
| Guo et al. | Gesture recognition for Chinese traffic police | |
| Devi et al. | Text extraction from images using gamma correction method and different text extraction methods—A comparative analysis | |
| Huang | A novel video text extraction approach based on Log-Gabor filters | |
| Ye et al. | Reading labels of cylinder objects for blind persons |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20180209 |