CN110647803A - Gesture recognition method, system and storage medium - Google Patents
Gesture recognition method, system and storage medium Download PDFInfo
- Publication number
- CN110647803A CN110647803A CN201910735382.5A CN201910735382A CN110647803A CN 110647803 A CN110647803 A CN 110647803A CN 201910735382 A CN201910735382 A CN 201910735382A CN 110647803 A CN110647803 A CN 110647803A
- Authority
- CN
- China
- Prior art keywords
- gesture
- radar
- matrix
- information
- gesture recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 239000011159 matrix material Substances 0.000 claims abstract description 95
- 230000009471 action Effects 0.000 claims abstract description 26
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 9
- 230000002159 abnormal effect Effects 0.000 claims description 7
- 230000009467 reduction Effects 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims description 4
- 230000001629 suppression Effects 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 description 13
- 238000005516 engineering process Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 8
- 230000003993 interaction Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000011176 pooling Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 206010011878 Deafness Diseases 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Human Computer Interaction (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Multimedia (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a gesture recognition method which is applied to a gesture recognition system comprising a first radar and a second radar, and the method comprises the steps of judging whether gesture actions exist in radar data or not; if the gesture action exists in the preset range, acquiring first gesture information and second gesture information; the first gesture information and the second gesture information respectively correspond to gesture information acquired by a first radar and a second radar; calculating to obtain a feature matrix according to the first gesture information and the second gesture information; the feature matrix comprises a first feature matrix and a second feature matrix corresponding to the first gesture information and the second gesture information; acquiring a gesture characteristic value according to the convolutional neural network; and recognizing the gesture according to the gesture value characteristics. The invention also provides a gesture recognition system and a storage medium. According to the invention, gesture recognition is carried out by two radars at the same time, so that the recognized gesture types can be increased, and the accuracy of gesture recognition is improved.
Description
Technical Field
The present invention relates to the field of computers, and in particular, to a gesture recognition method, system and storage medium.
Background
The gesture recognition technology is an important subject in the field of human-computer interaction, and aims to acquire hand action information through a sensor and recognize gesture types through an algorithm. The gesture action is an important link for communication in human daily life, and information which people want to express can be intuitively and effectively transmitted under a certain scene. Moreover, the gesture actions are rich in types and have specific significance, and the gesture actions can be directly used as a man-machine interaction mode to finish the interaction between a person and a computer. In addition, the gesture recognition technology can be fully applied to various aspects of human life, such as the functions of helping deaf-mutes to normally communicate, recognizing traffic police gestures in intelligent driving, controlling home work in intelligent home and the like. In short, gesture recognition technology, as the most natural man-machine interaction mode, will play a great role in future life. Gesture recognition can be seen as a way of computationally solving human language, building a richer bridge between machine and human than the original text user interface or even graphical user interface. Existing gesture recognition technologies include vision-based gesture recognition technologies, micro-point system-based gesture recognition technologies, and radar-based gesture recognition technologies. The gesture recognition technology based on the radar has a high data refresh rate, can work in various different environments, is not influenced by light, dust and the like, has a non-contact and natural privacy protection function, and will become an irreplaceable technology in the field of non-contact gesture recognition in the future. However, in the existing gesture recognition technology based on radar, when gesture information that can be collected by a single-station radar is adopted, the types of gestures that can be recognized are few, and there are many inconveniences in practical application.
Disclosure of Invention
The invention mainly aims to provide a gesture recognition method, a system and a storage medium, so as to increase the recognized gesture types.
In order to achieve the above object, the present invention provides a gesture recognition method applied to a gesture recognition system including a first radar and a second radar, the method including:
judging whether gesture actions exist in the acquired radar data; the radar data comprises radar data of a first radar and a second radar;
if the gesture action exists in the preset range, acquiring first gesture information and second gesture information; the first gesture information and the second gesture information respectively correspond to gesture information acquired by the first radar and the second radar;
calculating to obtain a gesture feature matrix according to the first gesture information and the second gesture information; the gesture feature matrix comprises a first feature matrix and a second feature matrix corresponding to the first gesture information and the second gesture information;
acquiring a gesture characteristic value according to the gesture characteristic matrix and the convolutional neural network;
and recognizing the gesture according to the gesture characteristic value.
Further, the "determining whether a gesture action exists in the radar data" includes:
detecting whether a gesture action exists according to the radar data;
if the gesture action exists, establishing plane coordinates by taking the first radar as an origin;
judging whether the gesture motion is in a gesture range or not according to the plane coordinate information;
if the gesture action is judged to be within the gesture range, acquiring first gesture information and second gesture information according to the radar data; the gesture range is a common detection range preset by the first radar and the second radar.
Further, the "calculating a gesture feature matrix according to the first gesture information and the second gesture information" includes:
respectively constructing a first matrix and a second matrix according to the first gesture information and the second gesture information;
removing abnormal values in the first matrix and the second matrix;
matrix overturning is carried out on the first matrix and the second matrix;
and completing the first matrix and the second matrix after the matrix inversion to obtain a first characteristic matrix and a second characteristic matrix.
Further, the "removing the abnormal value in the first matrix and the second matrix" includes:
and performing smoothing treatment on the row data of the first matrix and the second matrix according to a sliding window algorithm.
Further, the acquiring a gesture feature value according to the gesture feature matrix and the convolutional neural network includes:
performing convolution expansion on the first characteristic matrix and the second characteristic matrix respectively to obtain a first convolution and a second convolution;
connecting the first convolution and the second convolution expansion into a one-dimensional gesture characteristic value;
and recognizing the gesture according to the one-dimensional gesture characteristic value.
Further, before the step of determining whether the gesture motion exists in the preset range, the gesture recognition method further includes:
removing self interference signals of the first radar or the second radar receiving end;
and removing the external interference signal.
Further, the "removing the self interference signal of the first radar or the second radar receiving end" includes:
acquiring a gesture-free signal of a second preset time;
calculating self interference signals according to the gesture-free signals;
and removing self interference signals received by the first radar or the second radar.
Further, the "removing the external interference signal" specifically includes removing the external interference signal according to a clutter suppression method; the clutter suppression method comprises the following steps: linear phase FIR filtering method, adaptive average clutter reduction method, adaptive iteration clutter reduction method.
The invention also provides a gesture recognition system comprising a first radar and a second radar, the gesture recognition system further comprising a processor and a memory, the memory storing a gesture recognition program configured to be executed by the processor to implement the gesture recognition method described above.
The present invention also provides a storage medium, which is a computer-readable storage medium, and on which a gesture recognition program is stored, where the gesture recognition program is executable by one or more processors to implement the above gesture recognition method.
Compared with the prior art, the gesture recognition method has the advantages that the gesture feature matrix is obtained by the two radars at the same time, and the recognition is carried out through the convolutional neural network, so that the recognized gesture types can be increased, and the accuracy of gesture recognition is improved.
Drawings
Fig. 1 is a schematic structural diagram of a gesture recognition system according to an embodiment of the present invention;
FIG. 2 is a flowchart of a gesture recognition method according to an embodiment of the present invention;
FIG. 3 is a schematic view of a sub-flow of step S101 in FIG. 1;
FIG. 4 is a sub-flowchart of step S105 in FIG. 1;
FIG. 5 is a schematic view of a sub-flow of step S107 in FIG. 1;
fig. 6 is a comparison diagram of recognition accuracy of the gesture recognition method according to the embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the description relating to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
The embodiment of the present invention provides a gesture recognition method, which is applied to a gesture recognition system, please refer to fig. 1, the gesture recognition system includes a first radar 100, a second radar 200, a processor 300 and a memory 400, and the gesture recognition system further includes a network interface (not shown) and a communication bus 500. The processor 300 and the memory 400 are connected by a communication bus 500. The first radar 100 and the second radar 200 are connected by a communication bus 500. The memory 400 stores a gesture recognition program configured to be executed by the processor 300, and the processor 300 executes the gesture recognition program to implement the gesture recognition method. Referring to fig. 2, the gesture recognition method includes:
step S100: acquiring radar data of a first radar and a second radar within a preset range;
step S101: judging whether gesture actions exist in the radar data or not; if the gesture action exists in the preset range, executing step S103; otherwise, returning to execute the step S101;
step S103: acquiring first gesture information and second gesture information;
step S105: calculating to obtain a gesture feature matrix according to the first gesture information and the second gesture information;
step S107: acquiring a gesture characteristic value according to the gesture characteristic matrix and the convolutional neural network;
step S109: and recognizing the gesture according to the gesture characteristic value.
In this embodiment, the first gesture information and the second gesture information respectively correspond to gesture information acquired by the first radar and the second radar; the gesture feature matrix comprises a first feature matrix and a second feature matrix corresponding to the first gesture information and the second gesture information. Specifically, the transmitting terminals of the first radar 100 and the second radar 200 are aligned with the preset gesture range, and the reflected sound waves are received through the receiving terminals of the first radar 100 and the second radar 200. The gesture range is a range for acquiring the optimal gesture motion within the radar radiation range. In this embodiment, the signal collected by the radar may be represented as Rd(m,ni) Specifically, the signal collected by the first radar 100 can be represented as R1(m,ni) The signal collected by the second radar 200 is denoted as R2(m,ni). Wherein n isiIs a slow speed sample at time i, m is niFast time sampling of the sub-pulses.
Preferably, the radar receiving signal includes energy leakage caused by a radar antenna, and to eliminate the influence on the antenna, before step S101, the gesture recognition method further includes: removing the first radar 100 or the second radar200, receiving a self interference signal by a terminal; specifically, a gesture-free signal of a second preset time is collected; calculating self interference signals according to the gesture-free signals; and removing self interference signals received by the first radar or the second radar. It can be understood that the self-interference signal is obtained by acquiring a period of non-gesture signal, calculating the stable value of the signal in the non-gesture state by an averaging method, and calculatingSubtracting self interference signal from signal collected by radar to obtain signal Xd(m,ni). In particular, the method comprises the following steps of,
and removing the external interference signal.
Preferably, before step S101, during the gesture detection, there are many dynamic interference signals generated, which may be caused by micro-motion of other parts of the human body or other reasons, and these signals may be referred to as clutter Md(m,ni) And according to a linear phase FIR filtering method or a self-adaptive average clutter reduction method, suppressing external interference signals. The clutter suppressed signal is Fd(m,ni)。
Preferably, in order to confirm the timing of starting to recognize the gesture, it is required to detect whether the gesture exists within the radar range, specifically, referring to fig. 3, step S101 includes:
step S201: detecting whether a gesture action exists according to the radar data; if the gesture action exists, executing step S203; otherwise, returning to step S201;
step S203: establishing plane coordinates by taking the first radar as an origin;
step S205: judging whether the gesture motion is in a gesture range or not according to the plane coordinate information; if the gesture motion is determined to be within the gesture range, executing step S207; otherwise, returning to execute the step S201;
step S207: and acquiring first gesture information and second gesture information according to the radar data.
Specifically, whether a gesture target exists or not can be detected through a constant false alarm detector arranged in the radar, and whether the gesture is in a valid range or not is detected through one of the radars. In some embodiments, if the gesture is not within the gesture range, a prompt may be sent to make the recognition effect better.
Preferably, in step S103, the acquired first gesture information and the acquired second gesture information are gesture information respectively acquired by the first radar 100 and the second radar 200 within a first preset time; in the present embodiment, the first preset time is 2 seconds(s).
Preferably, referring to fig. 4, step S105 includes:
step S301: respectively constructing a first matrix and a second matrix according to the first gesture information and the second gesture information;
step S303: removing abnormal values in the first matrix and the second matrix;
step S305: matrix overturning is carried out on the first matrix and the second matrix;
step S307: and completing the first matrix and the second matrix after the matrix inversion to obtain a first characteristic value and a second characteristic matrix.
In particular, a first matrix T is constructed1Comprises the following steps:
and a second matrix T2Is as follows;
in order to avoid the existence of an abnormal value in the constructed matrix, the abnormal value in the matrix needs to be removed, and specifically, according to a sliding window algorithm, smoothing processing is performed on the matrix to realize the determination of each element value on a matrix row. And after matrix inversion is carried out on the matrix, the first matrix and the second matrix are filled up by a zero filling method. In this embodiment, the first matrix and the second matrix are filled to a matrix with a size of 128 × 128, and a first feature matrix and a second feature matrix are obtained.
Preferably, referring to fig. 5, the step S107 includes:
step S401: performing convolution expansion on the first characteristic matrix and the second characteristic matrix respectively to obtain a first convolution and a second convolution;
step S403: connecting the first convolution and the second convolution expansion into a one-dimensional gesture characteristic value;
step S405: and acquiring gesture characteristics according to the one-dimensional gesture characteristic value.
In this embodiment, the first feature matrix and the second feature matrix are respectively convolved three times, specifically, the first convolution process first performs 16 filters with a size of 3 × 3 and a step size of 2 to realize convolution, and then performs 2 × 2 maximum pooling layers with a step size of 2 and linear rectification activation convolution. The second convolution process is performed by 32 filters of size 3 x 3 with step size 1 and 64 filters of size 3 x 3 with step size 1, followed by 2 x 2 max pooling layers with step size 2 and linear rectification activated convolution. The third convolution process is the same as the second convolution process step. And performing convolution processing and expansion on the first feature matrix and the second feature matrix for three times respectively, outputting 8 x 64 obtained one-dimensional first convolution matrix and second convolution matrix respectively, expanding and connecting the two matrixes into a one-dimensional gesture characteristic value of 8192 x 1, inputting the one-dimensional gesture characteristic value into a normalized exponential function output by 14, and recognizing the gesture action.
Referring to fig. 6, fig. 6 is a schematic diagram illustrating comparison between recognition accuracy rates of a gesture recognition method and a single radar recognition method 1 and a single radar recognition method 2 in a self-adaptive learning process according to an embodiment of the present invention. The abscissa in fig. 6 represents the percentage of the training set, and the ordinate represents the recognition accuracy, and it can be seen from the figure that in the embodiment of the present invention, even when the training set is 30%, the recognition rate can still reach 96%, and as the proportion of the training set increases, the test accuracy is also improved, and when the training set reaches 70%, the test accuracy reaches about 98%. And the test accuracy rate of using one radar alone is far less than that of using two radars.
In the embodiment, two-dimensional coordinates can be measured by using two radars, and effective gestures are detected, so that the difficulty and interference influence of subsequent radar gesture recognition can be reduced; the dual-radar system can receive radar gesture signals reflected from different angles, has spatial angle and distance information, is beneficial to effective recognition of track gestures, and can learn the angle-distance information of the dual-station radar in a self-adaptive manner through a convolutional neural network. The invention can be applied to the field of man-machine interaction and control, can finish man-machine interaction in places with poor light and severe environment, such as the automobile running at night to control the media playing in the automobile and the like, and can also be applied to various control systems on intelligent home; the method can also be used for effective interaction and control in the environment with obstacles, and has important application value for indoor automation application.
In this embodiment, the memory 400 includes at least one type of readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, and the like. The memory 400 may be an internal storage unit of the gesture recognition system, such as a hard disk of the gesture recognition system, in some embodiments. The memory 400 may also be an external storage device of the gesture recognition system in other embodiments, such as a plug-in hard disk provided on the gesture recognition system, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and so on. Further, the memory 400 may also include both an internal storage unit of the gesture recognition system and an external storage device. The memory 400 may be used not only to store application software installed in the gesture recognition system and various types of data, such as codes of the gesture recognition system, etc., but also to temporarily store data that has been output or will be output.
The processor 300 may be, in some embodiments, a Central Processing Unit (CPU), controller, microcontroller, microprocessor or other data Processing chip for executing program codes stored in the memory 400 or Processing data, such as executing a gesture recognition program.
The network interface may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), typically used to establish a communication link between the gesture recognition system and other electronic devices.
The network interface is used to enable connectivity communications between these components.
A communication bus 500 is used to enable connectivity communication between these components.
In addition, an embodiment of the present invention further provides a storage medium, where the storage medium stores a gesture recognition program, and the gesture recognition program is executable by one or more processors to implement the following operations:
step S101: judging whether a gesture action exists in a preset range; if the gesture action exists in the preset range, executing step S103; otherwise, returning to execute the step S101;
step S103: acquiring first gesture information and second gesture information;
step S105: calculating to obtain a gesture feature matrix according to the first gesture information and the second gesture information;
step S107: acquiring a gesture characteristic value of the characteristic information according to the various matrixes of the gesture and the convolutional neural network;
step S109: and recognizing the gesture according to the gesture characteristic value.
The specific implementation of the storage medium of the present invention is substantially the same as the above-mentioned gesture recognition method and the gesture recognition embodiments, and will not be described herein in a repeated manner.
It should be noted that the above-mentioned numbers of the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments. And the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that comprises the element.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on this understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for causing a gesture recognition to perform the methods according to the embodiments of the present invention.
The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store or a data storage device, such as a server, a data center, etc., that is integrated with one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A gesture recognition method is applied to a gesture recognition system comprising a first radar and a second radar, and is characterized by comprising the following steps:
judging whether gesture actions exist in the acquired radar data; the radar data comprises radar data of a first radar and a second radar;
if the gesture action exists in the preset range, acquiring first gesture information and second gesture information; the first gesture information and the second gesture information respectively correspond to gesture information acquired by the first radar and the second radar;
acquiring a gesture feature matrix according to the first gesture information and the second gesture information; the gesture feature matrix comprises a first feature matrix and a second feature matrix corresponding to the first gesture information and the second gesture information;
acquiring a gesture characteristic value according to the gesture characteristic matrix and the convolutional neural network;
and recognizing the gesture according to the gesture characteristic value.
2. The gesture recognition method according to claim 1, characterized in that: the step of judging whether the gesture action exists in the radar data comprises the following steps:
detecting whether a gesture action exists according to the radar data;
if the gesture action exists, establishing plane coordinates by taking the first radar as an origin;
judging whether the gesture motion is in a gesture range or not according to the plane coordinate information;
if the gesture action is judged to be within the gesture range, acquiring first gesture information and second gesture information according to the radar data; the gesture range is a common detection range preset by the first radar and the second radar.
3. The gesture recognition method according to claim 2, characterized in that: the step of calculating and obtaining a gesture feature matrix according to the first gesture information and the second gesture information includes:
respectively constructing a first matrix and a second matrix according to the first gesture information and the second gesture information;
removing abnormal values in the first matrix and the second matrix;
matrix overturning is carried out on the first matrix and the second matrix;
and completing the first matrix and the second matrix after the matrix inversion to obtain a first characteristic matrix and a second characteristic matrix.
4. The gesture recognition method according to claim 3, characterized in that: the "removing the abnormal value in the first matrix and the second matrix" includes:
and performing smoothing treatment on the row data of the first matrix and the second matrix according to a sliding window algorithm.
5. The gesture recognition method according to claim 3, characterized in that: the step of acquiring the gesture eigenvalue according to the gesture eigenvalue matrix and the convolutional neural network comprises the following steps:
performing convolution expansion on the first characteristic matrix and the second characteristic matrix respectively to obtain a first convolution and a second convolution;
connecting the first convolution and the second convolution expansion into a one-dimensional gesture characteristic value;
and recognizing the gesture according to the one-dimensional gesture characteristic value.
6. The gesture recognition method according to claim 1, characterized in that: before the step of judging whether the gesture action exists in the preset range, the gesture recognition method further comprises the following steps:
removing self interference signals of the first radar or the second radar receiving end;
and removing the external interference signal.
7. The gesture recognition method according to claim 6, characterized in that: the step of removing the self interference signal of the first radar or the second radar receiving end comprises the following steps:
acquiring a gesture-free signal of a second preset time;
calculating self interference signals according to the gesture-free signals;
and removing self interference signals received by the first radar or the second radar.
8. The gesture recognition method of claim 7, wherein: the step of removing the external interference signal specifically comprises the step of removing the external interference signal according to a clutter suppression method; the clutter suppression method comprises the following steps: linear phase FIR filtering method, adaptive average clutter reduction method, adaptive iteration clutter reduction method.
9. A gesture recognition system comprising a first radar and a second radar, characterized by: the gesture recognition system further comprises a processor and a memory, the memory storing a gesture recognition program configured to be executed by the processor to implement the gesture recognition method of any one of claims 1-8.
10. A storage medium, characterized by: the storage medium is a computer readable storage medium, and the storage medium stores a gesture recognition program, and the gesture recognition program is executable by one or more processors to implement the gesture recognition method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910735382.5A CN110647803B (en) | 2019-08-09 | 2019-08-09 | Gesture recognition method, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910735382.5A CN110647803B (en) | 2019-08-09 | 2019-08-09 | Gesture recognition method, system and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110647803A true CN110647803A (en) | 2020-01-03 |
| CN110647803B CN110647803B (en) | 2023-12-05 |
Family
ID=68990091
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910735382.5A Active CN110647803B (en) | 2019-08-09 | 2019-08-09 | Gesture recognition method, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110647803B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111382702A (en) * | 2020-03-09 | 2020-07-07 | 深圳大学 | Gesture recognition method and device, mobile terminal and storage medium |
| CN111399642A (en) * | 2020-03-09 | 2020-07-10 | 深圳大学 | Gesture recognition method and device, mobile terminal and storage medium |
| CN111736688A (en) * | 2020-02-27 | 2020-10-02 | 珠海市杰理科技股份有限公司 | Bluetooth headset, system and gesture recognition method thereof |
| CN111813223A (en) * | 2020-07-09 | 2020-10-23 | 电子科技大学 | Gesture automatic segmentation and recognition method based on high-resolution radar signals |
| CN112099103A (en) * | 2020-09-14 | 2020-12-18 | 北京数衍科技有限公司 | Pedestrian information detection method and device and electronic equipment |
| CN112415510A (en) * | 2020-11-05 | 2021-02-26 | 深圳大学 | Double-station radar gesture recognition method, device and system and storage medium |
| CN113147669A (en) * | 2021-04-02 | 2021-07-23 | 淮南联合大学 | Gesture motion detection system based on millimeter wave radar |
| CN113220112A (en) * | 2020-01-21 | 2021-08-06 | 华为技术有限公司 | Gesture recognition method and device, terminal equipment and computer storage medium |
| CN114647302A (en) * | 2020-12-18 | 2022-06-21 | 开酷科技股份有限公司 | Gesture collection and recognition system with machine learning accelerator |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103455144A (en) * | 2013-08-22 | 2013-12-18 | 深圳先进技术研究院 | Vehicle-mounted man-machine interaction system and method |
| CN107024685A (en) * | 2017-04-10 | 2017-08-08 | 北京航空航天大学 | A kind of gesture identification method based on apart from velocity characteristic |
| CN107544061A (en) * | 2017-09-11 | 2018-01-05 | 深圳大学 | Knowledge based engineering is sparse to recover space-time adaptive processing method and system |
| US20180101239A1 (en) * | 2016-10-09 | 2018-04-12 | Alibaba Group Holding Limited | Three-dimensional graphical user interface for informational input in virtual reality environment |
| US20180157330A1 (en) * | 2016-12-05 | 2018-06-07 | Google Inc. | Concurrent Detection of Absolute Distance and Relative Movement for Sensing Action Gestures |
| CN109032349A (en) * | 2018-07-10 | 2018-12-18 | 哈尔滨工业大学 | A kind of gesture identification method and system based on millimetre-wave radar |
-
2019
- 2019-08-09 CN CN201910735382.5A patent/CN110647803B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103455144A (en) * | 2013-08-22 | 2013-12-18 | 深圳先进技术研究院 | Vehicle-mounted man-machine interaction system and method |
| US20180101239A1 (en) * | 2016-10-09 | 2018-04-12 | Alibaba Group Holding Limited | Three-dimensional graphical user interface for informational input in virtual reality environment |
| US20180157330A1 (en) * | 2016-12-05 | 2018-06-07 | Google Inc. | Concurrent Detection of Absolute Distance and Relative Movement for Sensing Action Gestures |
| CN107024685A (en) * | 2017-04-10 | 2017-08-08 | 北京航空航天大学 | A kind of gesture identification method based on apart from velocity characteristic |
| CN107544061A (en) * | 2017-09-11 | 2018-01-05 | 深圳大学 | Knowledge based engineering is sparse to recover space-time adaptive processing method and system |
| CN109032349A (en) * | 2018-07-10 | 2018-12-18 | 哈尔滨工业大学 | A kind of gesture identification method and system based on millimetre-wave radar |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113220112B (en) * | 2020-01-21 | 2023-07-18 | 华为技术有限公司 | Gesture recognition method, gesture recognition device, terminal equipment and computer storage medium |
| CN113220112A (en) * | 2020-01-21 | 2021-08-06 | 华为技术有限公司 | Gesture recognition method and device, terminal equipment and computer storage medium |
| CN111736688A (en) * | 2020-02-27 | 2020-10-02 | 珠海市杰理科技股份有限公司 | Bluetooth headset, system and gesture recognition method thereof |
| CN111399642A (en) * | 2020-03-09 | 2020-07-10 | 深圳大学 | Gesture recognition method and device, mobile terminal and storage medium |
| CN111382702B (en) * | 2020-03-09 | 2024-01-02 | 深圳大学 | Gesture recognition method, device, mobile terminal and storage medium |
| CN111382702A (en) * | 2020-03-09 | 2020-07-07 | 深圳大学 | Gesture recognition method and device, mobile terminal and storage medium |
| CN111399642B (en) * | 2020-03-09 | 2023-09-29 | 深圳大学 | Gesture recognition method and device, mobile terminal and storage medium |
| CN111813223A (en) * | 2020-07-09 | 2020-10-23 | 电子科技大学 | Gesture automatic segmentation and recognition method based on high-resolution radar signals |
| CN111813223B (en) * | 2020-07-09 | 2022-04-22 | 电子科技大学 | Gesture automatic segmentation and recognition method based on high-resolution radar signals |
| CN112099103A (en) * | 2020-09-14 | 2020-12-18 | 北京数衍科技有限公司 | Pedestrian information detection method and device and electronic equipment |
| CN112415510A (en) * | 2020-11-05 | 2021-02-26 | 深圳大学 | Double-station radar gesture recognition method, device and system and storage medium |
| CN114647302A (en) * | 2020-12-18 | 2022-06-21 | 开酷科技股份有限公司 | Gesture collection and recognition system with machine learning accelerator |
| CN113147669B (en) * | 2021-04-02 | 2022-08-02 | 淮南联合大学 | Gesture motion detection system based on millimeter wave radar |
| CN113147669A (en) * | 2021-04-02 | 2021-07-23 | 淮南联合大学 | Gesture motion detection system based on millimeter wave radar |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110647803B (en) | 2023-12-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110647803B (en) | Gesture recognition method, system and storage medium | |
| CN108596092B (en) | Gesture recognition method, device, equipment and storage medium | |
| CN102055836B (en) | Mobile terminal with action recognition function and action recognition method thereof | |
| US20200241112A1 (en) | Localization method and robot using the same | |
| CN108759892B (en) | Sensor calibration method, electronic device and computer-readable storage medium | |
| CN108710469A (en) | The startup method and mobile terminal and medium product of a kind of application program | |
| CN108181612B (en) | Determine the method and relevant apparatus of microphone beam profile angle | |
| CN110216661A (en) | Fall the method and device of region recognition | |
| CN110687513B (en) | Human body target detection method, device and storage medium | |
| CN110788866B (en) | Robot awakening method and device and terminal equipment | |
| CN111382702B (en) | Gesture recognition method, device, mobile terminal and storage medium | |
| US20200201968A1 (en) | User authentication control using ultrasound | |
| CN115047419A (en) | Constant false alarm rate detection method and device for millimeter wave radar | |
| CN115550615B (en) | Video recording method and device of intelligent equipment, electronic equipment and storage medium | |
| CN113129484B (en) | Door lock control method and device, automobile and storage medium | |
| CN112630736B (en) | Parameter determination method, device, equipment and storage medium of road side radar | |
| Abdelzaher et al. | Scheduling classifiers for real-time hazard perception considering functional uncertainty | |
| CN109358755B (en) | Gesture detection method and device for mobile terminal and mobile terminal | |
| CN109031192B (en) | Object positioning method, object positioning device and electronic equipment | |
| CN107329619B (en) | Method and device for triggering click event on touch terminal | |
| US20190163837A1 (en) | Digital data filtering method, apparatus, and terminal device | |
| CN114791705B (en) | Control method and device for household equipment, electronic equipment and readable storage medium | |
| CN115297205B (en) | Processing method, processing device, electronic equipment and storage medium | |
| CN117373056A (en) | Data processing method, system, terminal equipment and medium based on intelligent bathroom mirror | |
| CN116755441B (en) | Obstacle avoidance method, device, equipment and medium of mobile robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |