CN112286339B - Multi-dimensional gesture recognition device and method, electronic equipment and storage medium - Google Patents
Multi-dimensional gesture recognition device and method, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112286339B CN112286339B CN201910664913.6A CN201910664913A CN112286339B CN 112286339 B CN112286339 B CN 112286339B CN 201910664913 A CN201910664913 A CN 201910664913A CN 112286339 B CN112286339 B CN 112286339B
- Authority
- CN
- China
- Prior art keywords
- dimensional
- gesture
- dimensional gesture
- sensor
- recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention provides a multi-dimensional gesture recognition device, a multi-dimensional gesture recognition method, electronic equipment and a storage medium, wherein the device comprises a multi-dimensional gesture recognition module and a wireless communication module; the multi-dimensional gesture recognition module comprises four three-dimensional gesture sensors arranged in the left side, the top, the right side and the central direction of the device, a camera arranged in the middle of the device, an infrared laser arranged at the bottom of the device and a main control chip arranged in the device; the wireless communication module is used for sending the recognized gesture information to a user; the device integrates two-dimensional gesture recognition and a three-dimensional gesture recognition algorithm, and non-finger feature points can be effectively distinguished and filtered out by using the three-dimensional gesture to assist recognition of the two-dimensional gesture. The accuracy of the two-dimensional gesture and the robustness of the whole system are improved, and false recognition is reduced. Meanwhile, the device can identify the comprehensive gestures triggering the three-dimensional gesture sensors, can identify the comprehensive gestures into a group of actions, and provides richer identifiable user gestures.
Description
Technical Field
The invention belongs to the technical field of gesture control, and particularly relates to a multi-dimensional gesture recognition device, a multi-dimensional gesture recognition method, electronic equipment and a storage medium.
Background
At present, in the prior art, a plurality of methods are used for gesture recognition, for example, an infrared array module method is used for gesture recognition, the method cannot recognize two-dimensional gesture input in a plane, the recognition precision of three-dimensional gestures is low, and an infrared array is composed of a plurality of infrared light-emitting diodes and has high power consumption. It is inconvenient to debug and maintain. The infrared array is inconvenient to integrate due to the large size. For example, the method using an infrared laser and a monocular camera cannot recognize three-dimensional gesture input in a space. And the presence or absence of the characteristic points in the two-dimensional plane can only be identified, so that the lifting process and the falling process of the finger cannot be identified. This results in a lack of intermediate process information for feature points during the gesture motion. When other objects exist in the plane, the false recognition of the finger can be caused. And the scheme of the infrared laser and the monocular camera cannot effectively filter interference points of non-fingers. For example, a capacitive sensor scheme is adopted, and since the scheme belongs to a non-contact scheme, a two-dimensional gesture input of a contact type in a plane cannot be recognized. The capacitive sensor scheme has low accuracy for recognizing gestures at a longer distance. For example, leapmotion or Kinect, which requires an operating system and platform support. Has certain rigid requirements on an operation platform, has higher cost and large volume and is not easy to transplant. For example, in "a multi-mode dynamic gesture recognition system, device and method", two-dimensional and three-dimensional gesture recognition schemes are supported, although two gesture recognition methods, namely two-dimensional gestures and three-dimensional gestures, are supported. But are independent of each other and have no mutual dependency relationship. In the recognition algorithm, the situation that a plurality of sensors cooperate with each other is not supported. Even if the number of the sensors is simply increased, for example, the number of the sensors is increased to 3, only 3 independent gestures can be recognized, and the 3 independent gestures cannot be integrated into a complete action. In addition, the three-dimensional gesture recognition proposed therein is not integrated with its two-dimensional gesture recognition. It cannot help its two-dimensional gesture module to filter the interference points of non-fingers. And its communication mode adopts USB communication, and USB adopts wired connection, and is with higher costs and inconvenient than wireless communication.
Disclosure of Invention
The invention aims to solve the problems in the prior art and provides a multi-dimensional gesture recognition device, a multi-dimensional gesture recognition method, electronic equipment and a storage medium.
The invention is realized by the following technical scheme, and provides a multi-dimensional gesture recognition device which comprises a multi-dimensional gesture recognition module and a wireless communication module; the multi-dimensional gesture recognition module comprises four three-dimensional gesture sensors arranged in the left side, the top, the right side and the central direction of the device, a camera arranged in the middle of the device, an infrared laser arranged at the bottom of the device and a main control chip arranged in the device; the wireless communication module is arranged in the device and used for sending the gesture information identified by the multi-dimensional gesture identification module to a user; the main control chip is used for controlling the operation and processing operation of the multi-dimensional gesture recognition module and the wireless communication module;
the camera and the infrared laser are used for acquiring two-dimensional plane gestures; the left three-dimensional gesture sensor, the top three-dimensional gesture sensor and the right three-dimensional gesture sensor respectively and independently work to perform independent three-dimensional gesture recognition, and two or more adjacent three-dimensional gesture sensors in the three sensors work cooperatively to perform multi-sensor cooperative three-dimensional gesture recognition; and the central three-dimensional gesture sensor adopts a three-dimensional auxiliary two-dimensional gesture recognition method to recognize two-dimensional plane gestures.
Further, the central three-dimensional gesture sensor recognizes a two-dimensional plane gesture by using a three-dimensional auxiliary two-dimensional gesture recognition method, and the three-dimensional auxiliary two-dimensional gesture recognition method can distinguish invalid feature points of a non-hand-pointing direction, because the non-hand-pointing direction in the plane does not affect the central three-dimensional gesture sensor, specifically: when a user's hand performs planar two-dimensional gesture operation, the hand simultaneously enters a central three-dimensional gesture sensor recognition area, the central three-dimensional gesture sensor is simultaneously operated during the two-dimensional gesture operation, corresponding three-dimensional gestures are generated, the type of the two-dimensional gestures is judged according to the three-dimensional gestures, and if no three-dimensional gestures are recognized in the recognition area, non-hand pointing is judged.
Further, the three-dimensional gestures include six three-dimensional gestures of up, down, left, and right waving of a hand and forward approaching and backward departing.
Furthermore, the camera is a monocular camera.
Further, the multi-sensor collaborative three-dimensional gesture includes a wave gesture in an upper left, upper right, upper left right, upper right, upper right left direction.
The invention also provides an identification method of the multi-dimensional gesture identification device, which comprises the following steps:
step 1, initializing system hardware;
step 2, the camera and the infrared laser acquire two-dimensional plane gestures of hand images of a user, meanwhile, whether a central three-dimensional gesture sensor successfully identifies a three-dimensional gesture is judged, if the central three-dimensional gesture sensor successfully identifies the three-dimensional gesture, the step 3 is executed, and if the central three-dimensional gesture sensor unsuccessfully identifies the three-dimensional gesture, the step 4 is executed;
3, recognizing a two-dimensional plane gesture according to the recognized three-dimensional gesture by adopting a three-dimensional auxiliary two-dimensional gesture recognition method, if the recognition is successful, obtaining a three-dimensional auxiliary two-dimensional plane gesture type, and executing the step 5; if the gesture cannot be successfully identified as the two-dimensional plane gesture, the current gesture is an independent three-dimensional gesture, so that an independent three-dimensional gesture is obtained, and the step 5 is executed;
step 4, recognizing the multi-sensor cooperation three-dimensional gesture, judging whether the recognition of the multi-sensor cooperation three-dimensional gesture is successful or not, if so, obtaining the multi-sensor cooperation three-dimensional gesture, and if not, obtaining the independent three-dimensional gesture as the three-dimensional gesture, and executing step 5;
and 5, sending the gesture recognition result obtained in the step 3 or the step 4 to a user through a wireless communication module.
Further, after each three-dimensional gesture sensor except the central three-dimensional gesture sensor successfully identifies the three-dimensional gesture of the user, waiting for a certain time, and if the three-dimensional gesture sensors adjacent to the current three-dimensional gesture sensor successfully identify the same gesture in the gesture direction of the gesture successfully identified by the current three-dimensional gesture sensor in the time, considering the two gestures to be a multi-sensor cooperation three-dimensional gesture; if the adjacent three-dimensional gesture sensors do not recognize the same gesture within the waiting time, the current three-dimensional gesture is an independent three-dimensional gesture.
The invention also provides an electronic device, which comprises a memory and a processor;
the memory for storing a computer program;
the processor, when executing the computer program, is configured to perform a recognition method of the multi-dimensional gesture recognition apparatus.
The invention further provides a computer readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the identification method of the multi-dimensional gesture recognition apparatus is realized.
The invention has the beneficial effects that:
1. the device provided by the invention can solve the problem that non-finger points cannot be distinguished in a plane gesture recognition scheme. The accuracy of the plane gesture recognition is improved.
2. The device provided by the invention can operate a plurality of three-dimensional gesture sensors to work cooperatively, so that more gestures are expanded.
3. The device provided by the invention supports both independent two-dimensional and three-dimensional gestures and multi-sensor coordinated gestures.
4. The device provided by the invention does not need the support of an operating system, can work independently, and has high integration level and good portability.
5. The device provided by the invention adopts wireless communication without wire connection.
6. The device provided by the invention supports secondary development and is low in cost.
Drawings
FIG. 1 is a general block diagram of a multi-dimensional gesture recognition apparatus according to the present invention;
FIG. 2 is a schematic structural diagram of the multi-dimensional gesture recognition apparatus according to the present invention;
FIG. 3 is a flow chart of a multi-dimensional gesture recognition method according to the present invention;
FIG. 4 is a flow chart of a three-dimensional assisted two-dimensional gesture recognition method;
FIG. 5 is a flow chart of three-dimensional gesture recognition;
FIG. 6 is a schematic diagram of a left up swing gesture;
FIG. 7 is a schematic view of a top-left swipe gesture;
FIG. 8 is a schematic diagram of a right up swing gesture;
FIG. 9 is a schematic view of a right-up swipe gesture;
FIG. 10 is a top left and right swipe gesture;
FIG. 11 is a top right and left swipe gesture.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
With reference to fig. 1 and fig. 2, the present invention provides a multi-dimensional gesture recognition apparatus, which includes a multi-dimensional gesture recognition module and a wireless communication module; the multi-dimensional gesture recognition module comprises four three-dimensional gesture sensors arranged in the left side (not shown in fig. 2), the top, the right side and the central direction of the device, a camera arranged in the middle of the device, an infrared laser arranged at the bottom of the device and a main control chip arranged in the device; the device also contains necessary power supply, auxiliary circuit, connecting wire and the like. The wireless communication module is arranged in the device and used for sending the gesture information identified by the multi-dimensional gesture identification module to a user; the main control chip is used for controlling the operation and processing operation of the multi-dimensional gesture recognition module and the wireless communication module;
the camera and the infrared laser are used for collecting two-dimensional plane gestures; the left three-dimensional gesture sensor, the top three-dimensional gesture sensor and the right three-dimensional gesture sensor respectively work independently to perform independent three-dimensional gesture recognition, and two or more adjacent three-dimensional gesture sensors in the three sensors work cooperatively to perform multi-sensor cooperative three-dimensional gesture recognition; the central three-dimensional gesture sensor adopts a three-dimensional auxiliary two-dimensional gesture recognition method to recognize two-dimensional plane gestures.
The wireless communication module is used for sending the gesture recognition information to a user, the trouble of wired connection can be omitted by adopting wireless communication, the cost is reduced, multiple devices are supported to be connected, and the installation and debugging are convenient. The wireless communication module comprises a wireless communication controller, the main control chip is responsible for processing the successfully recognized gestures according to the format of a communication protocol, and then the wireless communication controller is called to send out data.
With reference to fig. 4, the central three-dimensional gesture sensor recognizes the two-dimensional plane gesture by using a three-dimensional auxiliary two-dimensional gesture recognition method, the three-dimensional auxiliary two-dimensional gesture recognition method can distinguish invalid feature points of a non-hand point, and the central three-dimensional gesture sensor is not affected by the non-hand point in the plane, so as to distinguish an effective finger gesture and an invalid non-hand point, the three-dimensional auxiliary two-dimensional gesture recognition method can effectively reduce the probability of false recognition and improve the accuracy of the gesture, and specifically comprises the following steps: when a user's hand performs planar two-dimensional gesture operation, the hand simultaneously enters a central three-dimensional gesture sensor recognition area, the central three-dimensional gesture sensor is simultaneously operated during the two-dimensional gesture operation, a current gesture image is obtained under the condition that the three-dimensional gesture is judged to exist, image preprocessing and contour extraction are performed, so that a corresponding three-dimensional gesture is generated by finding a feature point, the type of the two-dimensional gesture is judged according to the three-dimensional gesture, and if any three-dimensional gesture is not recognized in the recognition area, the hand is judged to be not pointed. The three-dimensional gesture is used for assisting the recognition of the two-dimensional gesture, so that the characteristic points of non-fingers can be effectively distinguished and filtered. The accuracy of the two-dimensional gesture is improved, and false recognition is reduced. And the robustness of the whole system is improved.
The three-dimensional gestures comprise six three-dimensional gestures of swinging up, down, left and right hands, and moving forwards to approach and backwards to get away. Each individual three-dimensional gesture sensor can recognize six three-dimensional gestures of up, down, left and right waving, forward approaching and backward departing of a hand.
The camera is a monocular camera. The monocular camera and the infrared laser are used for collecting two-dimensional plane gestures. The infrared laser is used for enhancing the brightness of the characteristic points in the identification area. The monocular camera is used for shooting images in the plane operation area, and infrared light can be reflected into the monocular camera after fingers enter the plane operation area. The main control chip is used for carrying out image processing, contour extraction and other calculations on the pictures acquired by the monocular camera, and further judging the two-dimensional plane gestures of the user.
The multi-sensor collaborative three-dimensional gesture includes a swipe gesture in an upper left, upper right, upper left right, upper right, and upper right left direction.
With reference to fig. 3, the present invention further provides an identification method of a multi-dimensional gesture recognition apparatus, including the following steps:
step 1, initializing system hardware;
step 2, the camera and the infrared laser acquire two-dimensional plane gestures of hand images of a user, meanwhile, whether a central three-dimensional gesture sensor successfully identifies a three-dimensional gesture is judged, if the central three-dimensional gesture sensor successfully identifies the three-dimensional gesture, the step 3 is executed, and if the central three-dimensional gesture sensor unsuccessfully identifies the three-dimensional gesture, the step 4 is executed;
3, recognizing a two-dimensional plane gesture according to the recognized three-dimensional gesture by adopting a three-dimensional auxiliary two-dimensional gesture recognition method, if the recognition is successful, obtaining a three-dimensional auxiliary two-dimensional plane gesture type, and executing the step 5; if the gesture cannot be successfully identified as the two-dimensional plane gesture, the current gesture is an independent three-dimensional gesture, so that an independent three-dimensional gesture is obtained, and the step 5 is executed;
step 4, recognizing the multi-sensor cooperation three-dimensional gesture, judging whether the recognition of the multi-sensor cooperation three-dimensional gesture is successful or not, if so, obtaining the multi-sensor cooperation three-dimensional gesture, and if not, obtaining the independent three-dimensional gesture, and executing step 5;
and 5, sending the gesture recognition result obtained in the step 3 or the step 4 to a user through a wireless communication module.
After each three-dimensional gesture sensor except the central three-dimensional gesture sensor successfully identifies the three-dimensional gesture of the user, waiting for a certain time, preferably 1 second, and if the same gesture is successfully identified by the three-dimensional gesture sensor adjacent to the current three-dimensional gesture sensor in the gesture direction of the gesture successfully identified by the current three-dimensional gesture sensor within the time, considering that the two gestures are a multi-sensor cooperation three-dimensional gesture; if the adjacent three-dimensional gesture sensors do not recognize the same gesture within the waiting time, the current three-dimensional gesture is an independent three-dimensional gesture. The method can identify the comprehensive gesture triggering the three-dimensional gesture sensors, can identify the comprehensive gesture into a group of actions, and provides richer identifiable user gestures.
Gesture recognition through the top sensor, starting with the left sensor or the right sensor, is exemplified. The specific identification process is shown in fig. 5, after the gesture of the left sensor is judged, whether the current gesture is a right waving gesture is judged, if yes, timing is started, the top sensor starts to identify the gesture, and if the gesture successfully identified by the top sensor is not overtime, the gesture is the multi-sensor cooperation three-dimensional gesture and is the same as the current gesture; if not, the current gesture is considered to be an independent three-dimensional gesture. The judgment modes of the right sensor and the top sensor are the same as those of the right sensor and the top sensor.
Multi-dimensional gesture recognition device based on stm32
Hardware list:
| name(s) | Type selection |
| Master control chip | stm32f407zgt6 |
| Three-dimensional gesture sensor | PAJ7620 Module |
| Camera head | ov5640 |
| Wireless communication controller | ESP8266 serial port module |
| Infrared laser | 808nm wavelength infrared word line |
Supported gestures:
the system comprises the following components: stm32f407zgt6 development board, PAJ7620 module (top sensor), PAJ7620 module (central sensor), ov5640 camera, 808nm wavelength infrared laser, ESP8266 wireless module, PAJ7620 module (left sensor) and PAJ7620 module (right sensor).
The definition of the multi-sensor collaborative gesture is as follows:
swing on the left top: the palm is swung in the direction of the arrow shown in fig. 6 to trigger the left and top three-dimensional gesture sensors.
Swinging up and left: the palm is swung in the direction of the arrow shown in fig. 7, triggering the top and left three-dimensional gesture sensors.
Swinging upwards on the right: the palm is swung in the direction of the arrow shown in FIG. 8 to trigger the right and top three-dimensional gesture sensors.
Swinging up and right: the palm is swung in the direction of the arrow shown in fig. 9, triggering the top and right three-dimensional gesture sensors.
Left-up-right swing: the palm is swung in the direction of the arrow shown in fig. 10 to trigger the left, top and right three-dimensional gesture sensors.
Right-up left swing: the palm is swung in the direction of the arrow shown in FIG. 11 to trigger the right, top and left three-dimensional gesture sensors.
The custom communication protocol is as follows:
| frame header | Length of | Gesture numbering | Ordinate/three-dimensional gesture subclass | Abscissa of the circle | Frame end |
| 0xA5 | 0x04 | n | y/m | x | 0x5A |
The third byte n represents the gesture number, and the relationship between the value of n and the gesture meaning is shown in table 1.
And when the value of n is 0x 01-0 x02, the plane gesture is represented, and the fourth byte is the vertical coordinate of the hand point in the plane.
And when the value of n is 0x 03-0 x06, the independent three-dimensional gesture of the corresponding sensor is represented, and at the moment, the fourth byte represents a subclass of the three-dimensional gesture.
And when the value of n is 0x 07-0 x0c, the multi-sensor cooperative three-dimensional gesture is represented.
TABLE 1n values vs. gesture meanings
| Value of n | Meaning of gesture |
| 0x01 | Single finger click |
| 0x02 | One finger slide |
| 0x03 | Left sensor |
| 0x04 | Top sensor |
| 0x05 | Right sensor |
| 0x06 | Central sensor |
| 0x07 | Swing from top left |
| 0x08 | Waving from the upper left |
| 0x09 | Swing from the upper right |
| 0x0a | Waving from the upper right |
| 0x0b | Waving from the upper left to the right |
| 0x0c | Right upper left waving |
When the value of n is 0x 03-0 x06, the independent three-dimensional gesture of the corresponding sensor is represented, m represents the subclass gesture, and the value of m is shown in table 2:
TABLE 2m values vs. gesture meanings
| Value of m | Meaning of gesture |
| 0x01 | Move upwards |
| 0x02 | Lower swing |
| 0x03 | Left waving |
| 0x04 | Right waving |
| 0x05 | Close to |
| 0x06 | Far away from |
The invention also provides an electronic device, which comprises a memory and a processor;
the memory for storing a computer program;
the processor, when executing the computer program, is configured to perform a recognition method of the multi-dimensional gesture recognition apparatus.
The invention further provides a computer readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the identification method of the multi-dimensional gesture recognition apparatus is realized.
The foregoing describes in detail a multi-dimensional gesture recognition apparatus, method, electronic device and storage medium, and specific examples are applied herein to explain the principles and embodiments of the present invention, and the descriptions of the foregoing embodiments are only used to help understand the method and its core ideas of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (8)
1. A multi-dimensional gesture recognition device, characterized by: the device comprises a multi-dimensional gesture recognition module and a wireless communication module; the multi-dimensional gesture recognition module comprises four three-dimensional gesture sensors arranged in the left side, the top, the right side and the central direction of the device, a camera arranged in the middle of the device, an infrared laser arranged at the bottom of the device and a main control chip arranged in the device; the wireless communication module is arranged in the device and used for sending the gesture information identified by the multi-dimensional gesture identification module to a user; the main control chip is used for controlling the operation and processing operation of the multi-dimensional gesture recognition module and the wireless communication module;
the camera and the infrared laser are used for collecting two-dimensional plane gestures; the left three-dimensional gesture sensor, the top three-dimensional gesture sensor and the right three-dimensional gesture sensor respectively and independently work to perform independent three-dimensional gesture recognition, and two or more adjacent three-dimensional gesture sensors in the three sensors work cooperatively to perform multi-sensor cooperative three-dimensional gesture recognition; the central three-dimensional gesture sensor adopts a three-dimensional auxiliary two-dimensional gesture recognition method to recognize two-dimensional plane gestures;
the central three-dimensional gesture sensor adopts a three-dimensional auxiliary two-dimensional gesture recognition method to recognize two-dimensional plane gestures, the three-dimensional auxiliary two-dimensional gesture recognition method can distinguish invalid characteristic points of non-hand pointing, and the central three-dimensional gesture sensor cannot be influenced by the non-hand pointing in the plane, and the method specifically comprises the following steps: when a user's hand performs planar two-dimensional gesture operation, the hand simultaneously enters a central three-dimensional gesture sensor recognition area, the central three-dimensional gesture sensor is simultaneously operated during the two-dimensional gesture operation, corresponding three-dimensional gestures are generated, the type of the two-dimensional gestures is judged according to the three-dimensional gestures, and if no three-dimensional gesture is recognized in the recognition area, the hand is judged to be not pointed.
2. The apparatus of claim 1, wherein: the three-dimensional gestures comprise six three-dimensional gestures of up, down, left and right swinging of hands, forward approaching and backward departing.
3. The apparatus of claim 1, wherein: the camera is a monocular camera.
4. The apparatus of claim 1, wherein: the multi-sensor collaborative three-dimensional gesture includes a swiping gesture in an upper left, upper right, upper left right, upper right, and upper right left direction.
5. A recognition method of the multi-dimensional gesture recognition device according to any one of claims 1-4, characterized by: the method comprises the following steps:
step 1, initializing system hardware;
step 2, the camera and the infrared laser acquire two-dimensional plane gestures of hand images of a user, meanwhile, whether a central three-dimensional gesture sensor successfully identifies a three-dimensional gesture is judged, if the central three-dimensional gesture sensor successfully identifies the three-dimensional gesture, the step 3 is executed, and if the central three-dimensional gesture sensor unsuccessfully identifies the three-dimensional gesture, the step 4 is executed;
3, recognizing a two-dimensional plane gesture according to the recognized three-dimensional gesture by adopting a three-dimensional auxiliary two-dimensional gesture recognition method, if the recognition is successful, obtaining a three-dimensional auxiliary two-dimensional plane gesture type, and executing the step 5; if the gesture cannot be successfully identified as the two-dimensional plane gesture, the current gesture is an independent three-dimensional gesture, so that an independent three-dimensional gesture is obtained, and the step 5 is executed;
step 4, recognizing the multi-sensor cooperation three-dimensional gesture, judging whether the recognition of the multi-sensor cooperation three-dimensional gesture is successful or not, if so, obtaining the multi-sensor cooperation three-dimensional gesture, and if not, obtaining the independent three-dimensional gesture, and executing step 5;
and 5, sending the gesture recognition result obtained in the step 3 or the step 4 to a user through a wireless communication module.
6. The method of claim 5, wherein: after each three-dimensional gesture sensor except the central three-dimensional gesture sensor successfully identifies the three-dimensional gesture of the user, waiting for a certain time, and if the three-dimensional gesture sensor adjacent to the current three-dimensional gesture sensor also successfully identifies the same gesture in the gesture direction successfully identified by the current three-dimensional gesture sensor in the time, considering the two gestures to be a multi-sensor cooperation three-dimensional gesture; if the adjacent three-dimensional gesture sensors do not recognize the same gesture within the waiting time, the current three-dimensional gesture is an independent three-dimensional gesture.
7. An electronic device, characterized in that: the electronic device comprises a memory and a processor;
the memory for storing a computer program;
the processor, for when executing the computer program, being configured to perform the identification method of claim 5 or 6.
8. A computer-readable storage medium, characterized in that the storage medium has stored thereon a computer program which, when being executed by a processor, carries out the identification method as claimed in claim 5 or 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910664913.6A CN112286339B (en) | 2019-07-23 | 2019-07-23 | Multi-dimensional gesture recognition device and method, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910664913.6A CN112286339B (en) | 2019-07-23 | 2019-07-23 | Multi-dimensional gesture recognition device and method, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112286339A CN112286339A (en) | 2021-01-29 |
| CN112286339B true CN112286339B (en) | 2022-12-16 |
Family
ID=74418769
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910664913.6A Active CN112286339B (en) | 2019-07-23 | 2019-07-23 | Multi-dimensional gesture recognition device and method, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112286339B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101901106A (en) * | 2009-05-26 | 2010-12-01 | 美国智能科技有限公司 | The method and the device that are used for the data input |
| CN102971701A (en) * | 2010-06-17 | 2013-03-13 | 高通股份有限公司 | Methods and apparatus for contactless gesture recognition and power reduction |
| CN103547989A (en) * | 2011-04-13 | 2014-01-29 | 诺基亚公司 | A method, apparatus and computer program for user control of a state of an apparatus |
| CN103713735A (en) * | 2012-09-29 | 2014-04-09 | 华为技术有限公司 | Method and device of controlling terminal equipment by non-contact gestures |
| CN103729054A (en) * | 2012-10-10 | 2014-04-16 | 三星电子株式会社 | Multi display device and control method thereof |
| CN105912100A (en) * | 2016-03-31 | 2016-08-31 | 智车优行科技（北京）有限公司 | Screensaver lifting device, screensaver lifting method and vehicle |
| CN108011996A (en) * | 2016-11-01 | 2018-05-08 | 谷歌公司 | Control outputting and inputting on more sides of computing device |
| CN108027691A (en) * | 2015-06-04 | 2018-05-11 | 华为技术有限公司 | Determine input equipment, user equipment and the method for movement |
| CN109656375A (en) * | 2019-02-28 | 2019-04-19 | 哈尔滨拓博科技有限公司 | A kind of multi-mode dynamic hand gesture recognition system, device and method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101844366B1 (en) * | 2009-03-27 | 2018-04-02 | 삼성전자 주식회사 | Apparatus and method for recognizing touch gesture |
| CN104932692B (en) * | 2015-06-24 | 2017-12-08 | 京东方科技集团股份有限公司 | Three-dimensional tactile method for sensing, three-dimensional display apparatus, wearable device |
-
2019
- 2019-07-23 CN CN201910664913.6A patent/CN112286339B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101901106A (en) * | 2009-05-26 | 2010-12-01 | 美国智能科技有限公司 | The method and the device that are used for the data input |
| CN102971701A (en) * | 2010-06-17 | 2013-03-13 | 高通股份有限公司 | Methods and apparatus for contactless gesture recognition and power reduction |
| CN103547989A (en) * | 2011-04-13 | 2014-01-29 | 诺基亚公司 | A method, apparatus and computer program for user control of a state of an apparatus |
| CN103713735A (en) * | 2012-09-29 | 2014-04-09 | 华为技术有限公司 | Method and device of controlling terminal equipment by non-contact gestures |
| CN103729054A (en) * | 2012-10-10 | 2014-04-16 | 三星电子株式会社 | Multi display device and control method thereof |
| CN108027691A (en) * | 2015-06-04 | 2018-05-11 | 华为技术有限公司 | Determine input equipment, user equipment and the method for movement |
| CN105912100A (en) * | 2016-03-31 | 2016-08-31 | 智车优行科技（北京）有限公司 | Screensaver lifting device, screensaver lifting method and vehicle |
| CN108011996A (en) * | 2016-11-01 | 2018-05-08 | 谷歌公司 | Control outputting and inputting on more sides of computing device |
| CN109656375A (en) * | 2019-02-28 | 2019-04-19 | 哈尔滨拓博科技有限公司 | A kind of multi-mode dynamic hand gesture recognition system, device and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112286339A (en) | 2021-01-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11373054B2 (en) | Object recognition method and mobile terminal | |
| EP3754542B1 (en) | Method and apparatus for recognizing handwriting in air, and device and computer-readable storage medium | |
| US9671872B2 (en) | Gesture recognition method, gesture recognition system, terminal device and wearable device | |
| TWI659331B (en) | Screen capture method and device for smart terminal | |
| CN104350509A (en) | Fast pose detector | |
| WO2015003606A1 (en) | Method and apparatus for recognizing pornographic image | |
| CN111708366A (en) | Robot, method and device for controlling action of robot, and computer-readable storage medium | |
| WO2017070971A1 (en) | Facial authentication method and electronic device | |
| CN112507918B (en) | Gesture recognition method | |
| CN104834412A (en) | Touch terminal based on non-contact hand gesture recognition | |
| KR20150106823A (en) | Gesture recognition apparatus and control method of gesture recognition apparatus | |
| US20150049035A1 (en) | Method and apparatus for processing input of electronic device | |
| CN103336950A (en) | Human face identification method and system | |
| CN107291238B (en) | Data processing method and device | |
| CN112286339B (en) | Multi-dimensional gesture recognition device and method, electronic equipment and storage medium | |
| CN115826764B (en) | Gesture control method and system based on thumb | |
| US9176588B2 (en) | System and method for discerning complex gestures using an array of optical sensors | |
| US20230133783A1 (en) | Method for detecting active pen touch position, and electronic terminal | |
| CN108021882B (en) | Face recognition system of robot based on time interval | |
| CN116301551A (en) | Touch identification method, touch identification device, electronic equipment and medium | |
| CN106920208A (en) | A kind of face stylizing method and electronic equipment based on image procossing | |
| CN103576906B (en) | mouse icon control method and system | |
| WO2022022365A1 (en) | Image matching method applied to fingerprint recognition and related device | |
| CN110991307B (en) | Face recognition method, device, equipment and storage medium | |
| JP5456817B2 (en) | Display control apparatus, display control method, information display system, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |