CN112783326A - Gesture recognition device and gesture recognition system - Google Patents
Gesture recognition device and gesture recognition system Download PDFInfo
- Publication number
- CN112783326A CN112783326A CN202110117652.3A CN202110117652A CN112783326A CN 112783326 A CN112783326 A CN 112783326A CN 202110117652 A CN202110117652 A CN 202110117652A CN 112783326 A CN112783326 A CN 112783326A
- Authority
- CN
- China
- Prior art keywords
- information
- gesture
- gesture recognition
- user
- recognition device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013186 photoplethysmography Methods 0.000 claims abstract description 68
- 210000000707 wrist Anatomy 0.000 claims abstract description 60
- 230000009471 action Effects 0.000 claims abstract description 42
- 230000033001 locomotion Effects 0.000 claims abstract description 32
- 238000001514 detection method Methods 0.000 claims abstract description 19
- 238000004891 communication Methods 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 12
- 230000005540 biological transmission Effects 0.000 claims description 10
- 210000001367 artery Anatomy 0.000 claims description 6
- 210000004247 hand Anatomy 0.000 abstract description 8
- 238000000034 method Methods 0.000 description 24
- 210000002435 tendon Anatomy 0.000 description 15
- 230000008569 process Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 11
- 210000002321 radial artery Anatomy 0.000 description 11
- 210000002559 ulnar artery Anatomy 0.000 description 11
- 230000003287 optical effect Effects 0.000 description 7
- 230000002093 peripheral effect Effects 0.000 description 7
- 210000003811 finger Anatomy 0.000 description 6
- 206010011878 Deafness Diseases 0.000 description 5
- 230000003993 interaction Effects 0.000 description 5
- 238000013473 artificial intelligence Methods 0.000 description 4
- 238000006073 displacement reaction Methods 0.000 description 4
- 210000003414 extremity Anatomy 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 230000009429 distress Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000029058 respiratory gaseous exchange Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000012512 characterization method Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 210000003813 thumb Anatomy 0.000 description 2
- 230000002792 vascular Effects 0.000 description 2
- 230000003321 amplification Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 210000003423 ankle Anatomy 0.000 description 1
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000009351 contact transmission Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 210000005224 forefinger Anatomy 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000010438 heat treatment Methods 0.000 description 1
- 210000001503 joint Anatomy 0.000 description 1
- 210000004932 little finger Anatomy 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 229910052760 oxygen Inorganic materials 0.000 description 1
- 239000001301 oxygen Substances 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000035485 pulse pressure Effects 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
Abstract
The invention discloses a gesture recognition device and a gesture recognition system, wherein the gesture recognition device comprises: the terminal is provided with a gesture detection module and a processor, wherein the gesture detection module is used for acquiring first information, and the first information is used for representing the gesture movement track of a user; the wrist strap is connected with the terminal and is provided with at least two first photo-plethysmography (PPG) sensors which are used for acquiring second information, and the second information is used for representing biological sensing signals of the hand of a user; the processor is used for acquiring the first information and the second information and recognizing the gesture action of the user according to the first information and the second information. The gesture recognition device provided by the invention has the advantages that the recognition mode does not depend on image recognition, the accuracy of gesture recognition can be improved, the two hands of a user are not required to be placed in the recognition area specified by the camera, and the user experience is more relaxed.
Description
Technical Field
The present invention relates to the field of gesture recognition technologies, and in particular, to a gesture recognition apparatus and a gesture recognition system.
Background
Gesture recognition is a research hotspot in the field of human-computer interaction, the current gesture recognition is mainly realized through image recognition, and a gesture recognition device acquires a video or an image of a gesture action of a user by using a camera, and then recognizes the gesture action by using algorithm analysis processing. Because the gesture action range is small and the action time is short, the image precision requirement for gesture recognition is obviously higher than that for recognizing the actions of other limbs of the body, and the image recognition is not easy to realize in some application scenes, such as scenes with insufficient light in a carriage, dark places such as a movie theater and the like, multi-person scenes with complex environmental backgrounds, scenes in which the gesture cannot be captured from the front side in a camera, and the like.
Disclosure of Invention
The present invention is directed to solving at least one of the problems of the prior art. Therefore, the invention provides a gesture recognition device and a gesture recognition system, which can improve the accuracy of gesture motion recognition.
In one aspect, an embodiment of the present invention provides a gesture recognition apparatus, including:
the terminal is provided with a gesture detection module and a processor, wherein the gesture detection module is used for acquiring first information, and the first information is used for representing the gesture movement track of a user;
a wrist band connected to the terminal, the wrist band being provided with at least two first photoplethysmography (PPG) sensors for acquiring second information representing a bio-sensing signal of a user's hand;
the processor is used for acquiring the first information and the second information and recognizing the gesture action of the user according to the first information and the second information.
The gesture recognition device provided by the embodiment of the invention at least has the following beneficial effects: gesture recognition device includes terminal and wrist area, be provided with gesture detection module and treater in the terminal, be provided with two at least first PPG sensors on the wrist area, acquire first information through gesture detection module, first information is that the gesture action orbit that is used for the characterization user, acquire the second information through first PPG sensor, the second information can be the biosensing signal who is used for the characterization user's hand, the biosensing signal of hand can include wrist ulnar artery or radial artery biosensing signal and wrist tendon biosensing signal, the treater handles first information, the second information, in order to accomplish the gesture recognition process.
The conventional gesture recognition device adopts a camera to recognize gestures, and has a plurality of limitations in the application process, for example, the gesture recognition device is difficult to work normally in a low-light environment and a dark environment. The two hands of a user are not required to be placed in the identification area specified by the camera, so that the two hands can move freely, the user experience is relaxed, the method is particularly suitable for man-machine interaction and the like during game operations such as AR and VR, the application scenes are wider, and the method can be used for assisting communication of deaf-mutes, controlling the artificial limbs of the disabled, controlling unmanned planes or robots, controlling exoskeleton equipment and the like.
According to some embodiments of the present invention, the terminal is further provided with a second PPG sensor, the second PPG sensor is configured to acquire third information, the third information is used to represent a bio-sensing signal of a hand of a user corresponding to the position of the terminal, and the processor is specifically configured to identify a gesture action of the user according to the first information, the second information, and the third information. The second PPG sensor of the combined terminal identifies the biological sensing signals of the tendons in the middle of the wrist, so that the accuracy of gesture identification can be improved.
According to some embodiments of the invention, the wrist strap is further provided with a pressure sensor, the pressure sensor is used for acquiring fourth information, the fourth information is used for representing a hand artery signal of the user, and the processor is further used for carrying out wearing reminding on the user according to the fourth information. The pressure sensor is used for acquiring the wrist artery pulse signal, so that the user can be reminded of accurately wearing the gesture recognition device, and the gesture recognition action of the gesture recognition device is more accurate.
According to some embodiments of the invention, a PPG control unit is also provided for controlling the operating state of the first PPG sensor in dependence on control signals from the processor. According to the use demand, the PPG control unit can control the work of the first PPG sensor in a flexible way.
According to some embodiments of the invention, the terminal is detachably connected to the wrist strap, the terminal is provided with a first contact point, the wrist strap is provided with a second contact point, and the first contact point and the second contact point are electrically connected in a state in which the terminal is mounted in connection with the wrist strap. Terminal and wrist strap can pass through first contact and second contact transmission data, and gesture recognition device can design for terminal and wrist strap demountable installation, and under the condition that the terminal needs to charge, can charge for the terminal through first contact, hoisting device's use flexibility also is convenient for maintain and change under the condition that certain part damages.
According to some embodiments of the invention, an analog signal processing module is further provided, and the analog signal processing module is configured to collect the second information and transmit the second information to the processor. The second information is analog signals, and the analog signal processing module collects the second information and transmits the second information to the processor, so that the processor can process and recognize gesture actions conveniently.
According to some embodiments of the invention, a playing device is further provided, and the playing device is configured to play gesture action information recognized by the processor according to the first information and the second information. The playing device can be a display screen, a loudspeaker or a buzzer, the gesture recognition device recognizes the gesture action of the user, and the video image and voice corresponding to the gesture are played through the playing device of the gesture recognition device.
According to some embodiments of the invention, the terminal is provided with a wireless transmission unit, and the gesture recognition device is in communication connection with an external device through the wireless transmission unit. The gesture recognition device is internally provided with a wireless transmission unit which can be conveniently connected with other external equipment, the external equipment can be a mobile phone, a Bluetooth sound box, VR equipment or artificial exoskeleton and other equipment, the gesture recognition device recognizes gesture actions of a human body, can interact with the external equipment, performs other operations on the external equipment, can display pictures on the external equipment, can read voice or operate the external equipment to act, can effectively improve use experience, and the wireless transmission mode can be Bluetooth connection, Zigbee protocol connection or other connection modes.
In another aspect, an embodiment of the present invention provides a gesture recognition system, including: the electronic equipment is in communication connection with the gesture recognition device, and is used for playing gesture action information from the gesture recognition device.
The gesture recognition system provided by the embodiment of the invention at least has the following beneficial effects: the gesture recognition system comprises a gesture recognition device and an electronic device, wherein the electronic device is in wireless connection with the gesture recognition device of the first aspect, and the electronic device comprises a display screen or a loudspeaker. The gesture recognition device recognizes gestures through an algorithm, combines hand motion track information of the gesture detection module, outputs the gestures through natural language, can display images or characters on electronic equipment connected with the gesture recognition device, can read out voice through a loudspeaker of the electronic equipment, or executes other operations on a gesture recognition system, and can effectively expand application scenes, such as human-computer interaction, AR game control, VR game control, robot control and the like.
According to some embodiments of the invention, the electronic device is provided with a playing module, and the playing module is used for playing the gesture action information. The playing module can be a display screen, a loudspeaker or a buzzer, the gesture recognition device recognizes the gesture action of a user, and the video image, voice or video corresponding to the gesture is played through the playing module of the electronic equipment.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which the abstract is to be fully consistent with one of the figures of the specification:
FIG. 1 is a schematic diagram of a gesture recognition apparatus according to an embodiment of the present invention;
FIG. 2 is a schematic block diagram of a gesture recognition apparatus according to an embodiment of the present invention;
FIG. 3 is a waveform diagram of a gesture recognition apparatus according to another embodiment of the present invention;
FIG. 4 is a system architecture diagram of a gesture recognition apparatus according to another embodiment of the present invention;
fig. 5 is a schematic diagram of a gesture recognition apparatus according to another embodiment of the invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
In the description of the present invention, it should be understood that the orientation or positional relationship referred to in the description of the orientation, such as the upper, lower, front, rear, left, right, etc., is based on the orientation or positional relationship shown in the drawings, and is only for convenience of description and simplification of description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present invention.
In the description of the present invention, the meaning of a plurality of means is one or more, the meaning of a plurality of means is two or more, and larger, smaller, larger, etc. are understood as excluding the number, and larger, smaller, inner, etc. are understood as including the number. If the first and second are described for the purpose of distinguishing technical features, they are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
In the description of the present invention, unless otherwise explicitly limited, terms such as arrangement, installation, connection and the like should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific contents of the technical solutions.
The invention provides a gesture recognition device which comprises a terminal and a wrist strap, wherein the terminal is provided with a gesture detection module and a processor, the wrist strap is provided with at least two first PPG sensors, image recognition can be independent, normal work can be realized in a low-light environment, the gesture recognition accuracy is higher, the requirement that the two hands of a user are placed in a recognition area specified by a camera is avoided, the two hands of the user move more freely, and the user experience is relaxed.
The embodiments of the present invention will be further explained with reference to the drawings.
Referring to fig. 1, fig. 2, fig. 3 and fig. 4, fig. 1 is a schematic diagram of a gesture recognition apparatus 100 according to an embodiment of the present invention, fig. 2 is a schematic block diagram of the gesture recognition apparatus 100 according to another embodiment of the present invention, fig. 3 is a waveform diagram of the gesture recognition apparatus 100 according to another embodiment of the present invention, and fig. 4 is a system architecture diagram of the gesture recognition apparatus 100 according to another embodiment of the present invention.
In an embodiment, the gesture recognition apparatus 100 includes a terminal 110 and a wrist strap 120, a gesture detection module 430 and a processor 450 are disposed in the terminal 110, a first PPG sensor 131 is disposed on the wrist strap, first information is acquired by the gesture detection module 430, the first information is used for representing a gesture motion track of a user, second information is acquired by the first PPG sensor 131, the second information may be a bio-sensing signal used for representing a hand of the user, the bio-sensing signal of the hand may include a ulnar artery or radial artery bio-sensing signal and a tendon bio-sensing signal, and the processor 450 processes the first information and the second information to complete a gesture recognition process.
Photoplethysmography (PPG) is a widely used technique for obtaining physiological parameters of a human body, such as pulse, respiration, blood oxygen, etc., by using an optical method. In short, generally speaking, the light of the light emitting diode is used to irradiate the skin, and the photodiode is used to measure the quantity of light transmitted or reflected, so as to obtain a graph representing the volume change caused by the pulse pressure.
Although various physiological characteristic parameters can be obtained by the sensing technology, the sensing technology is susceptible to hand movement and generates corresponding movement artifacts. Currently, corresponding algorithms are developed for removing these motion artifacts. These algorithms look for optical intensity and time characteristics resulting from motion and reject the effects of relevant characteristics, thereby preserving parameters of physiological characteristics such as heartbeat or respiration.
The conventional application method is to eliminate noise information such as optical intensity and time characteristic influence generated by motion, so as to keep accurate parameters of physiological characteristics such as heartbeat or respiration, whereas the gesture recognition device 100 of the embodiment needs to consider the noise information and amplify and recognize the noise of the motion artifact, so as to recognize the gesture.
The first PPG sensor 131 arranged on the wrist band acquires second information, the second information is a biological sensing signal from the hand of a user, the second information is an optical signal, the analog signal processing module 420 acquires the optical signal and transmits the optical signal to the processor 450, the processor 450 selects dynamic characteristics of the second information by using an algorithm, for example, a gesture action with a sequence number of 1 in fig. 3 is performed, in this embodiment, a waveform diagram obtained by four first PPG sensors 131 is arranged on the wrist band, the user performs a gesture of straightening a thumb, an index finger and a little finger, other fingers are retracted, a waveform obtained by the first PPG sensor 131 with the sequence number of 1 sequentially has a high peak and two sub-peaks with similar heights, a waveform obtained by the first PPG sensor 131 with the sequence number of 2 sequentially has a high peak and a sub-peak, a waveform obtained by the first PPG sensor 131 with the sequence number of 3 sequentially has two peaks with a certain distance, the waveform obtained by the first PPG sensor 131 with sequence number 4 has three peaks with similar heights, two slightly smaller valleys are formed between the three peaks with similar heights, and the gesture checking module arranged at the terminal 110 acquires that the hand has no displacement motion, and the relevant gesture is recognized by grabbing the features of the waveform in the actual gesture recognition process.
Like the gesture action that the serial number is 2 of fig. 3, the diastole expands thumb and forefinger, other fingers are packed up, the waveform that first PPG sensor 131 that the serial number is 1 obtained has two highly similar secondary peaks and a peak in proper order, the waveform that first PPG sensor 131 that the serial number is 2 obtained has a secondary peak and a peak slightly higher in proper order, the waveform that first PPG sensor 131 that the serial number is 3 obtained has two highly similar secondary peaks and a peak slightly higher in proper order, the waveform that first PPG sensor 131 that the serial number is 4 obtained has a comparatively smooth wave, there is a little arch in the middle part of wave, and what set up that the gesture check module at terminal 110 obtained is that the hand does not have the displacement action, snatch through the characteristic to foretell waveform in order to discern relevant gesture in the gesture identification process of reality.
As shown in fig. 3, the gesture motion with sequence number 3 is a gesture motion using a ratio V motion of an index finger and a middle finger, the waveform obtained by the first PPG sensor 131 with sequence number 1 sequentially has a high peak, a low peak and a high peak, the waveform obtained by the first PPG sensor 131 with sequence number 2 sequentially has two high peaks that are close to each other, the waveform obtained by the first PPG sensor 131 with sequence number 3 sequentially has a high peak after a relatively flat line, the waveform obtained by the first PPG sensor 131 with sequence number 4 has a relatively smooth wave, a slight recess is formed in the middle of the wave, and the gesture checking module arranged at the terminal 110 obtains no displacement motion of the hand, and the gesture recognition process is performed by grabbing the features of the waveform to recognize the related gesture in the actual gesture recognition process.
Like the gesture action that the serial number is 4 of fig. 3, for stretching five fingers and do the action of shaking from side to side, the waveform that first PPG sensor 131 that the serial number is 1 obtained has a peak, the waveform that first PPG sensor 131 that the serial number is 2 obtained has a peak, the waveform that first PPG sensor 131 that the serial number is 3 obtained has a secondary peak and two peaks that are close in height in proper order, the waveform that first PPG sensor 131 that the serial number is 4 obtained has a peak that appears after fluctuating line from top to bottom, and it has the displacement action to set up what the gesture check module at terminal 110 acquireed is the hand, in order to discern relevant gesture through snatching to the characteristic of foretell waveform in the gesture identification process of reality.
The dynamic characteristics of the hand actions are selected by using an algorithm, and the hand action tracks detected by the gesture detection device are combined, so that the gesture actions can be well recognized, recognized gestures can be learned, the recognition accuracy of the gestures can be further improved in the using process, some preset oscillograms or dynamic characteristics can be input through external software upgrading, the gesture recognition range of the gesture recognition device 100 is increased, the gesture recognition device 100 can be more intelligent in the using process of a user, and the experience of the user can be better.
It should be noted that the 4 first PPG sensors 131 of this embodiment may be placed at the positions shown in fig. 1, but the positions shown in fig. 1 do not limit the placement positions and the number of the first PPG sensors 131 of this embodiment, and other numbers of first PPG sensors 131 may also be used.
Compared with the conventional gesture recognition equipment through image recognition, the gesture recognition device 100 of the embodiment can be better used even in a dark environment without a camera and ambient light, and the two hands of the user do not need to be placed in a designated recognition area, so that the two hands of the user can move more freely, and the use process is more relaxed. The application range of the embodiment is wide, the method is suitable for man-machine interaction during game operations of AR, VR and the like, and the method can be expanded to more applications which cannot be realized by utilizing image recognition, such as auxiliary communication of deaf-mutes, control of artificial limbs of disabled people, control of unmanned planes or robots, control of exoskeleton equipment and the like.
It should be noted that the gesture detection module 430 may be a module such as a gyroscope, a gravity accelerometer, a magnetometer, and the like, and may be implemented to detect a gesture motion trajectory, which is not limited in this embodiment.
In an embodiment, a second PPG sensor 132 is provided at the terminal 110, and the second PPG sensor 132 is used to acquire third information, which is used to characterize the midwrist tendon biosensing signal. The user correctly wears gesture recognition device 100, and terminal 110 can be and press close to the wrist back, and the identification area of the second PPG sensor 132 that sets up on terminal 110 can be wrist middle part tendon position, and second PPG sensor 132 can acquire the motion information of wrist middle part tendon, combines the motion trail that gesture detection module 430 obtained, and processor 450 carries out mode recognition to first information, second information, third information, can effectively promote gesture recognition's accuracy.
In one embodiment, the wrist strap 120 is provided with a pressure sensor 440, and the pressure sensor 440 is used to acquire fourth information, which is used to characterize the wrist artery pulse signal. Pressure sensor 440 can provide supplementary artery locate function, and pressure sensor 440's setting can remind the user to correctly wear gesture recognition device 100, and under the condition of correctly wearing, pressure sensor 440 can obtain clear artery pulse signal, can cooperate pronunciation or characters suggestion user's wearing methods, also can remind the user through the buzzer vibration that sets up at gesture recognition device 100. It should be noted that the number of the pressure sensors 440 may be one, or may be multiple, and the setting of the multiple pressure sensors 440 can receive the arterial pulse signal from different angles, so as to improve the accuracy of pressure identification and improve the user experience.
In an embodiment, the wrist band 120 is provided with two first PPG sensors 131, one of the first PPG sensors 131 may be used to acquire a biosensing signal from a wrist tendon, the other first PPG sensor 131 may be oriented towards a wrist ulnar artery or a radial artery to acquire a clearer vascular biosensing signal from the wrist ulnar artery or the radial artery, the first information may include the biosensing signal from the wrist tendon and the clearer vascular biosensing signal from the wrist ulnar artery or the radial artery, and in combination with the first information acquired by the gesture detection module 430, the first information is to identify a trajectory of a gesture motion, and the processor 450 acquires the first information and the second information and identifies the gesture motion of the user according to the first information and the second information.
It should be noted that the number of the first PPG sensors 131 for acquiring the biosensing signals from the wrist tendons may also be two, which may be disposed on both sides of the terminal 110, since tendons are grown on both sides of the wrist of the human body, the tendons on both sides can deform to some extent during the hand movement, the biosensing signals of the tendons on both sides of the wrist in the hand movement can be acquired, the first PPG sensors 131 for acquiring the biosensing signals of the wrist tendons are disposed on both sides, the reliability of the biosensing signal identification of the wrist tendons can be further improved, and the ability of the gesture identification device 100 to learn the gesture identification can also be improved. However, the number of the first PPG sensors 131 for acquiring the biosensing signals from the carpus tendons mentioned in the present embodiment is not limited to two, and may be three, four, five or even more.
It should be noted that the number of the first PP G sensors 131 for acquiring the biosensing signals from the ulnar artery or the radial artery of the wrist may also be two, and the first PP G sensors may be disposed on both sides of the terminal 110, because the ulnar artery or the radial artery of the wrist of the human body are arranged side by side, the ulnar artery or the radial artery may deform to some extent during the hand movement, and the two first PPG sensors 131 can better acquire the biosensing signals of the ulnar artery or the radial artery of the wrist due to the hand movement, so that the reliability of identifying the biosensing signals of the ulnar artery or the radial artery of the wrist can be further improved, and the capability of the gesture recognition device 100 to learn gesture recognition can also be improved. However, the number of the first PPG sensors 131 for acquiring biosensing signals from the ulnar artery or the radial artery of the wrist mentioned in the embodiment is not limited to two, and may be three, four, five or even more.
In an embodiment, the gesture recognition apparatus 100 is provided with a PPG control unit 410, the PPG control unit 410 is used to control a working state of the first PPG sensor 131, according to a use requirement, the PPG control unit 410 can flexibly control the first PPG sensor 131 to work, and the PPG control unit 410 may be disposed in the terminal 110 or disposed in the wrist band 120.
Referring to fig. 5, fig. 5 is a schematic diagram of a gesture recognition apparatus 100 according to another embodiment of the present embodiment.
In an embodiment, the terminal 110 is provided with a first contact 510 and the wrist strap 120 is provided with a second contact 520, the terminal 110 and wrist strap 120 being connected by the first contact 510 and the second contact 520. The terminal 110 and the wrist band 120 may be integrally connected or detachably connected, the terminal 110 and the wrist band 120 are connected through a first contact 510 and a second contact 520, the first PPG sensor 131 of the wrist band 120 is connected with the second contact 520, and the first PPG sensor 131 is connected to the terminal 110 through the first contact 510, so that communication between the first PPG sensor 131 and the processor 450 is realized. The design that terminal 110 and wrist strap 120 can dismantle the connection can be convenient for the wrist-watch through first contact 510510 charge, also can be convenient for the maintenance and the change of wrist strap 120, promotes the variety that wrist strap 120 selected, can select different wrist straps 120 according to the demand of difference.
In an embodiment, the gesture recognition apparatus 100 is provided with an analog signal processing module 420, and the analog signal processing module 420 is configured to collect the second information and transmit the second information to the processor 450. The first PPG sensor 131 is a sensor that acquires a wrist bio-sensing signal of the user, which is mainly an optical intensity and time characteristic generated due to movement. The main functions of the analog signal processing module 420 include the following aspects: signal amplification, frequency conversion, modulation, demodulation, adjacent frequency processing, level adjustment and control, and mixing, wherein the analog biological signal is processed by the analog signal processing module 420, so that the processor 450 selects motion characteristics by using an algorithm, the gesture detection module 430 is combined with the acquisition of the hand motion trajectory information, then the artificial intelligence algorithm is applied to recognize gestures, and after the processor 450 recognizes the gestures, natural sentences are applied to express the recognized gestures, and the recognized gestures can be displayed or played on the terminal 110 of the gesture recognition device 100, or played or displayed through an external device.
The artificial intelligence algorithm mentioned in this embodiment is a common processing method for gesture recognition, and the gesture recognition process is introduced through the artificial intelligence algorithm in the text "unmanned aerial vehicle gesture recognition design research under artificial intelligence background" of guo welcome and indian volition, which is not described in this embodiment again.
In an embodiment, the gesture recognition apparatus 100 is further provided with a playing device, and the playing device is used for playing the gesture recognition information of the gesture recognition apparatus 100. The playing device may be a display screen disposed on the terminal 110 facing the user, after the gesture recognition device 100 recognizes the gesture action of the user, the processor 450 displays the gesture action on the display screen in a self-heating language, image or video manner, one application field of the embodiment is to assist communication for the deaf-mute, the communication manner for the deaf-mute is generally sign language, but most people cannot understand sign language, and after the gesture recognition device 100 recognizes the gesture action of the user, the people communicating with the user can understand the meaning of the user through the content displayed on the display screen on the terminal 110.
Certainly, the playing module is not limited to the display screen, and may further include a speaker, a buzzer, and a vibration motor, and the sign language of the user may be translated into a natural voice through the speaker of the gesture recognition device 100, so as to transmit the meaning to be expressed by the user to other people in a sound manner, and enable communication between other people and the user to be smoother. When a user is in a dangerous state, the user performs a preset gesture action, the gesture recognition device 100 recognizes the distress information of the user, and then sends out a sound signal for distress through a buzzer on the terminal 110, of course, the gesture recognition device 100 also has a conversation function, and when the distress signal of the user is recognized, the user can give an alarm through one key of the gesture recognition device 100, and can communicate with police officers through the gesture action after the telephone is connected, so that the user can perform the gesture action, the gesture recognition device 100 recognizes gesture meanings, and then the meanings to be expressed are communicated to the police officers through a voice mode.
In an embodiment, the terminal 110 is provided with a wireless transmission unit, the wireless transmission unit is arranged in the gesture recognition device 100, and the wireless transmission unit can be conveniently in communication connection with other peripheral devices, where the peripheral devices may be a mobile phone, a bluetooth sound, a VR device or an artificial exoskeleton, and the like, and also can be used in scenes of auxiliary communication for deaf-mutes, control of artificial limbs for disabled people, and the like. The gesture recognition device 100 recognizes a gesture of a human body, may interact with a peripheral device, performs other operations on the peripheral device, may display a picture on the peripheral device, or read a voice or operate the peripheral device to perform an operation, and may effectively improve a use experience, where the wireless transmission mode may be a bluetooth connection, a Zigbee protocol Zigbee connection, or another connection mode.
The embodiment also provides a gesture recognition system, which includes an electronic device and the gesture recognition apparatus 1001 of the first aspect, where the electronic device is wirelessly connected to the gesture recognition apparatus 100 of the first aspect. The electronic device can receive gesture information recognized by the gesture recognition apparatus 100, and the electronic device includes but is not limited to: cell-phone, wearing equipment, unmanned vehicles, computer, panel computer, TV set, refrigerator, car, intelligence lock etc..
In an embodiment, an electronic device is wirelessly connected with the gesture recognition apparatus 100 of the first aspect, and the electronic device may include a display screen or a speaker. The gesture recognition device 100 recognizes gestures through an algorithm, combines hand motion trajectory information of the gesture detection module 430, and outputs the gestures through a natural language, and may display images or characters on an electronic device connected to the gesture recognition device 100, read out voice through a speaker of the electronic device, or perform other operations on the electronic device, so that application scenarios, such as human-computer interaction, unmanned machine control, AR game control, VR game control, robot control, and the like, can be effectively expanded.
For example, in a scenario of using an exoskeleton, a user may wear the gesture recognition device 100 and an artificial exoskeleton, which is often applied in a carrying scenario of some heavy objects, and the user may control the exoskeleton to move forward through a preset gesture action, or may control the exoskeleton to jump, move backward, accelerate, and the like through one gesture action, so that the control of the user on a peripheral device can be effectively improved. Of course, the gesture recognition apparatus 100 may be connected to the robot through wireless communication, and the user may control the robot through gesture operation.
For example, unmanned aerial vehicle control, gesture recognition device 100 and unmanned aerial vehicle wireless communication are connected, the user can control unmanned aerial vehicle to take off through raising the arm, reduce arm control unmanned aerial vehicle and descend, also can advance, retreat, accelerate and become operation such as looks through some other gesture motion control unmanned aerial vehicle, compare in the mode that current conventionality passes through handle control, the control mode of this embodiment is convenient more comfortable, experience sense is also better.
For example, AR game control and VR game control, the immersive experience of the game is always the direction that the user pursues, the gesture recognition device 100 provided in this embodiment can recognize the gesture motion of the user, the user may not only wear the gesture recognition device 100 provided in this embodiment, but also wear the gesture recognition device 100 provided in this embodiment on both left and right hands, or wear the gesture recognition device 100 provided in this embodiment on the ankle of the user, and the structure of the gesture recognition device 100 provided in this embodiment may not only be limited to the structure shown in fig. 1, but also be a piece of wear clothes designed in the shape of a protection arm, or a piece of clothes that can be worn closely, and a plurality of PPG sensors and gesture monitoring devices may be provided at various positions of the wrist, elbow, arm joint, chest, leg joint, and the like of the clothes, the actions of the user are restored to the actions of the body of the role in the game through the detection and identification of the actions of all parts of the body of the user, so that the control of the user on the game role can be improved very well, and the experience completely different from the existing game control mode is brought. Of course, the control method mentioned in this embodiment is not limited to be applied to a game scene, and may be applied to control of a robot having a large human-shaped structure.
It should be noted that, what kind of gesture is used to control what kind of operation is performed by the external device in this embodiment is only an example, and does not limit the control method in this embodiment.
The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention.
Claims (10)
1. A gesture recognition apparatus, comprising:
the terminal is provided with a gesture detection module and a processor, wherein the gesture detection module is used for acquiring first information, and the first information is used for representing the gesture movement track of a user;
a wrist band connected to the terminal, the wrist band being provided with at least two first photoplethysmography (PPG) sensors for acquiring second information representing a bio-sensing signal of a user's hand;
the processor is used for acquiring the first information and the second information and recognizing the gesture action of the user according to the first information and the second information.
2. The gesture recognition device according to claim 1, wherein the terminal is further provided with a second PPG sensor, the second PPG sensor is configured to obtain third information, the third information is used to represent a biosensing signal of a hand of the user corresponding to the position of the terminal, and the processor is specifically configured to recognize a gesture action of the user according to the first information, the second information, and the third information.
3. The gesture recognition device of claim 1, wherein the wrist strap is further provided with a pressure sensor, the pressure sensor is configured to obtain fourth information, the fourth information is used for representing a hand artery signal of the user, and the processor is further configured to remind the user to wear according to the fourth information.
4. The gesture recognition device according to claim 1, wherein a PPG control unit is further provided for controlling an operating state of the first PPG sensor according to the control signal from the processor.
5. Gesture recognition apparatus according to claim 1, characterized in that the terminal is detachably connected to the wrist strap, the terminal being provided with a first contact point and the wrist strap being provided with a second contact point, the first contact point and the second contact point being electrically connected in a state in which the terminal is mounted in connection with the wrist strap.
6. The gesture recognition device according to claim 1, further comprising an analog signal processing module, wherein the analog signal processing module is configured to collect the second information and transmit the second information to the processor.
7. The gesture recognition device according to claim 1, further comprising a playing device, wherein the playing device is configured to play gesture action information recognized by the processor according to the first information and the second information.
8. The gesture recognition device according to any one of claims 1 to 7, wherein the terminal is provided with a wireless transmission unit, and the gesture recognition device is in communication connection with an external device through the wireless transmission unit.
9. A gesture recognition system, comprising:
the electronic equipment and the gesture recognition device according to any one of claims 1 to 8, wherein the electronic equipment is in communication connection with the gesture recognition device and is used for playing gesture action information from the gesture recognition device.
10. The gesture recognition system of claim 9, wherein the electronic device is provided with a playback module for playing the gesture action information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110117652.3A CN112783326A (en) | 2021-01-28 | 2021-01-28 | Gesture recognition device and gesture recognition system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110117652.3A CN112783326A (en) | 2021-01-28 | 2021-01-28 | Gesture recognition device and gesture recognition system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112783326A true CN112783326A (en) | 2021-05-11 |
Family
ID=75759368
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110117652.3A Pending CN112783326A (en) | 2021-01-28 | 2021-01-28 | Gesture recognition device and gesture recognition system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112783326A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024006603A1 (en) * | 2022-06-29 | 2024-01-04 | Google Llc | Gesture detection via image capture of subdermal tissue from a wrist-pointing camera system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103581428A (en) * | 2012-07-27 | 2014-02-12 | Lg电子株式会社 | Terminal and control method thereof |
| WO2015123771A1 (en) * | 2014-02-18 | 2015-08-27 | Sulon Technologies Inc. | Gesture tracking and control in augmented and virtual reality |
| CN204965329U (en) * | 2014-09-30 | 2016-01-13 | 苹果公司 | Portable electronic equipment and motion and gesture detection system |
| US20160305875A1 (en) * | 2015-04-15 | 2016-10-20 | Pixart Imaging Inc. | Action recognizition system and method thereof |
| CN106774818A (en) * | 2015-11-20 | 2017-05-31 | 三星电子株式会社 | Posture identification method, gesture recognition device and wearable device |
| US20180338720A1 (en) * | 2017-05-25 | 2018-11-29 | Samsung Electronics Co., Ltd. | Method and wearable device for performing actions using body sensor array |
-
2021
- 2021-01-28 CN CN202110117652.3A patent/CN112783326A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103581428A (en) * | 2012-07-27 | 2014-02-12 | Lg电子株式会社 | Terminal and control method thereof |
| WO2015123771A1 (en) * | 2014-02-18 | 2015-08-27 | Sulon Technologies Inc. | Gesture tracking and control in augmented and virtual reality |
| CN204965329U (en) * | 2014-09-30 | 2016-01-13 | 苹果公司 | Portable electronic equipment and motion and gesture detection system |
| US20160305875A1 (en) * | 2015-04-15 | 2016-10-20 | Pixart Imaging Inc. | Action recognizition system and method thereof |
| CN106774818A (en) * | 2015-11-20 | 2017-05-31 | 三星电子株式会社 | Posture identification method, gesture recognition device and wearable device |
| US20180338720A1 (en) * | 2017-05-25 | 2018-11-29 | Samsung Electronics Co., Ltd. | Method and wearable device for performing actions using body sensor array |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024006603A1 (en) * | 2022-06-29 | 2024-01-04 | Google Llc | Gesture detection via image capture of subdermal tissue from a wrist-pointing camera system |
| US11934586B2 (en) | 2022-06-29 | 2024-03-19 | Google Llc | Gesture detection via image capture of subdermal tissue from a wrist-pointing camera system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11281262B2 (en) | Detecting a gesture made by a person wearing a wearable electronic device | |
| KR102265934B1 (en) | Method and apparatus for estimating ppg signal and stress index using a mobile terminal | |
| US11032457B2 (en) | Bio-sensing and eye-tracking system | |
| US20180116607A1 (en) | Wearable monitoring device | |
| EP2772828B1 (en) | Individual body discrimination device and individual body discrimination method | |
| KR20150061231A (en) | Digital device and method for controlling the same | |
| US11771375B2 (en) | Respiration rate detection device and breath detection device adopting motion denoising | |
| CN109069032A (en) | System and method for determining blood pressure | |
| US11484208B2 (en) | Attached sensor activation of additionally-streamed physiological parameters from non-contact monitoring systems and associated devices, systems, and methods | |
| CN110134245A (en) | A kind of eye control device and eye prosecutor method based on EOG and attitude transducer | |
| CN109222927A (en) | A kind of processing method based on health status, intelligent wearable device and storage medium | |
| KR20190098806A (en) | A smart hand device for gesture recognition and control method thereof | |
| Martínez-Villaseñor et al. | Deep learning for multimodal fall detection | |
| Motti et al. | Introduction to wearable computers | |
| KR20200078795A (en) | Apparatus and method for estimating blood pressure | |
| CN112783326A (en) | Gesture recognition device and gesture recognition system | |
| KR102562817B1 (en) | Electronic device for measuring blood pressure and operating method thereof | |
| US20210085259A1 (en) | Apparatus and method for estimating bio-information | |
| Gollan et al. | Making Sense: Experiences with Multi-Sensor Fusion in Industrial Assistance Systems. | |
| KR20220087658A (en) | Electronic apparatus and operation method for providing of workout guide thereof | |
| KR102323576B1 (en) | System for Determining Dementia Using Public Wireless Medical Device Based on IoT | |
| WO2022111704A1 (en) | Heart rate measurement method and electronic device | |
| KR20220003887A (en) | A method and an apparatus for estimating blood pressure | |
| KR102397934B1 (en) | A method and an apparatus for estimating blood pressure using an acceleration sensor and a gyro sensor | |
| CN114287938B (en) | Method and equipment for obtaining safety interval of human body parameters in building environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |