CN111190489B - Flexible wearable hand motion gesture recognition system and method and sensor - Google Patents
Flexible wearable hand motion gesture recognition system and method and sensor Download PDFInfo
- Publication number
- CN111190489B CN111190489B CN201911406527.3A CN201911406527A CN111190489B CN 111190489 B CN111190489 B CN 111190489B CN 201911406527 A CN201911406527 A CN 201911406527A CN 111190489 B CN111190489 B CN 111190489B
- Authority
- CN
- China
- Prior art keywords
- layer
- cloth
- chipless
- wireless
- flexible
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000033001 locomotion Effects 0.000 title claims abstract description 31
- 238000000034 method Methods 0.000 title claims abstract description 24
- 230000009471 action Effects 0.000 claims abstract description 11
- 239000010410 layer Substances 0.000 claims description 132
- 239000004744 fabric Substances 0.000 claims description 71
- 210000001145 finger joint Anatomy 0.000 claims description 11
- 238000004891 communication Methods 0.000 claims description 10
- 239000004753 textile Substances 0.000 claims description 10
- 238000005452 bending Methods 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 7
- 230000008859 change Effects 0.000 claims description 6
- 230000001681 protective effect Effects 0.000 claims description 6
- 230000000712 assembly Effects 0.000 claims description 5
- 238000000429 assembly Methods 0.000 claims description 5
- 230000001133 acceleration Effects 0.000 claims description 4
- 239000012790 adhesive layer Substances 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 claims description 4
- 230000005057 finger movement Effects 0.000 claims description 2
- 238000001514 detection method Methods 0.000 abstract description 40
- 230000005540 biological transmission Effects 0.000 abstract description 25
- 238000005516 engineering process Methods 0.000 abstract description 3
- 230000006872 improvement Effects 0.000 abstract description 3
- 210000003811 finger Anatomy 0.000 description 27
- 238000010586 diagram Methods 0.000 description 10
- 230000011218 segmentation Effects 0.000 description 8
- 238000004458 analytical method Methods 0.000 description 4
- 230000001419 dependent effect Effects 0.000 description 4
- 239000003292 glue Substances 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 210000001503 joint Anatomy 0.000 description 3
- 230000008054 signal transmission Effects 0.000 description 3
- 230000003321 amplification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000003199 nucleic acid amplification method Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 238000009941 weaving Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000005674 electromagnetic induction Effects 0.000 description 1
- 239000011888 foil Substances 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000010030 laminating Methods 0.000 description 1
- 210000004932 little finger Anatomy 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 210000000811 metacarpophalangeal joint Anatomy 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 238000005476 soldering Methods 0.000 description 1
- 239000002344 surface layer Substances 0.000 description 1
- 210000004243 sweat Anatomy 0.000 description 1
- 210000003813 thumb Anatomy 0.000 description 1
- 210000000707 wrist Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K17/00—Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations
- G06K17/0022—Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations arrangements or provisious for transferring data to distant stations, e.g. from a sensing device
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- User Interface Of Digital Computer (AREA)
- Arrangements For Transmission Of Measured Signals (AREA)
Abstract
The invention relates to a flexible wearable hand motion gesture recognition system, method and sensor, and belongs to the technical field of sensing technology and robot control. The hand action gesture recognition system comprises an upper computer for action gesture recognition, a glove worn on a monitored object, an RFID read-write module and a wireless chipless sensing assembly distributed on the glove; the wireless chipless sensing assembly comprises a flexible strain sensor and an RFID tag module, wherein the flexible strain sensor is arranged in the knuckle sleeve part, and the RFID tag module is connected with a connecting lead in parallel with the flexible strain sensor; the knuckle sleeve part is at least used for being sleeved outside a knuckle between two knuckle joints on a finger; on the same glove, the difference of the resistance values between any two flexible strain sensors is larger than a preset value. The structure of the sensor on the glove is effectively simplified through the structural improvement based on the flexible sensing assembly, wireless chipless sensing detection can be realized, transmission wiring is reduced, and the method can be widely applied to the technical fields of robot control, wireless chipless detection and the like.
Description
Technical Field
The invention relates to the technical field of hand motion gesture recognition, in particular to a flexible wearable hand motion gesture recognition system, a hand motion gesture recognition method and a flexible stretching sensor for constructing the hand motion gesture recognition system.
Background
With the development of wearable technology, especially the application of flexible stretchable sensors in wearable technology, a new solution is provided for the field of human motion gesture detection. The flexible stretchable sensor can be used as a detection device, can convert external physical deformation into electric signal quantity and output the electric signal quantity, and has good application prospect in the fields of electronic skin, health monitoring, flexible display screens, motion entertainment, detection sensing and the like.
Patent document No. CN109793295A discloses an intelligent glove, in which flexible conductive sheets are arranged on the surface of a glove body, any two flexible conductive sheets are not connected to each other and form an open circuit interface in a conductive loop, when a finger moves to make any two flexible conductive sheets contact, a path is formed in the conductive loop of the two flexible conductive sheets, so as to recognize the bending gesture of the finger.
In order to solve the above problems, an RFID sensor is usually used to construct a detection sensor, but a plurality of detection chips and a coil module for wireless power transmission still exist, so that the whole structure is complex and is difficult to be practically applied; furthermore, it is mainly used to address detection schemes in hard surface environments.
Disclosure of Invention
The invention mainly aims to provide a hand action gesture recognition method based on a wearable flexible sensing assembly, which effectively simplifies the structure of a detection sensor on a glove based on the structural improvement of the flexible sensing assembly, can realize wireless chipless sensing detection and effectively reduce signal transmission wiring;
another objective of the present invention is to provide a hand motion gesture recognition system based on a wearable flexible sensing assembly, so as to effectively simplify the structure of the detection sensor on the glove based on the structural improvement of the flexible sensing assembly, and to realize wireless chipless sensing detection and effectively reduce signal transmission wiring;
it is still another object of the present invention to provide a strain sensing assembly that can be used to construct the hand motion gesture recognition system.
In order to achieve the above main object, the hand motion gesture recognition method provided by the present invention comprises the steps of:
a transmitting step, namely transmitting an inquiry radio frequency signal to a wireless chipless flexible sensing assembly arranged on the glove; the sensing assembly comprises a flexible strain sensor arranged in each knuckle sleeve part of the glove and an RFID label module connected with a lead in parallel with the flexible strain sensor; the knuckle sleeve part provided with the sensing assembly is at least used for being sleeved outside a knuckle positioned between two knuckles on a finger; on the same glove, the difference between the resistance values of any two flexible strain sensors is larger than a preset value;
a receiving step of receiving a radio frequency signal fed back by an antenna;
and a step of identification, which is to output a hand action gesture identification result by utilizing the trained hand action gesture identification network model based on the received radio frequency signal.
The strain gauge with the resistance changing along with the deformation state of the strain gauge is connected in parallel to the RFID antenna to construct the wireless chipless sensor for detecting the bending/straightening state of the finger knuckle, so that the complex structures of related signal transmission line wiring, a wireless power coil, a control chip and the like can be reduced, and the structure of the detection sensor on the glove is effectively simplified.
The specific scheme is that the glove comprises a cloth inner layer, a cloth outer layer and an elastic support ring supported between the cloth inner layer and the cloth outer layer; the elastic support ring is composed of a plurality of annular rubber layers which are sleeved and fixedly connected into an integral structure, and the number of the layers is more than five and is an odd number; from inside to outside, the elastic support ring comprises a connecting layer part formed by alternately sleeving a high-elasticity rubber layer and a low-elasticity rubber layer, and a bottom high-elasticity rubber layer for fixedly connecting the inner cloth layer and the connecting layer part, wherein the bottom high-elasticity rubber layer is fixedly connected with the outer surface of the inner cloth layer in an adhering manner, and the elastic rubber layer fixedly connected with the outer cloth layer in an adhering manner is a low-elasticity rubber layer; the elastic support ring is tightly sleeved between the two finger joints and is positioned beside the outer end side finger joint; the wireless chipless flexible sensing component is arranged in the overhead layer structure supported by the elastic support ring and is fixedly connected with the outer surface of the inner layer of the cloth in an adhering mode.
Through laying the elastic support ring, the inlayer cloth that not only is convenient for lay the foil gage is straightened in advance when the finger straightens and is arranged, and can effectively avoid the removal of outer cloth to influence the operating condition of sensor.
The preferred scheme is that the inner layer of the cloth is an elastic textile cloth layer, the outer layer of the cloth is a waterproof textile cloth layer, and a protective adhesive layer is coated on the outer surface of the wireless chipless flexible sensing assembly.
The preferred scheme is that the knuckle sleeve part provided with the sensing assembly is only used for being sleeved outside a knuckle positioned between two knuckles on a finger; in the identifying step, the motion state of the end knuckle is a motion state drivenly coupled to the middle knuckle. Thereby effectively reducing the complexity of the modeling.
In order to achieve the other object, the hand motion gesture recognition system provided by the invention comprises an upper computer for motion gesture recognition, a glove worn on a monitored object, an RFID read-write module in communication connection with the upper computer, and a wireless chipless sensing assembly arranged on each finger sleeve part of the glove; the wireless chipless sensing assembly comprises a flexible strain sensor and an RFID tag module, wherein the flexible strain sensor is arranged in a knuckle sleeve part of the finger sleeve part, and the RFID tag module is connected with a connecting lead in parallel with the flexible strain sensor; the knuckle sleeve part provided with the wireless chipless sensing assembly is at least used for being sleeved outside a knuckle between two knuckles on a finger; on the same glove, the difference of the resistance values between any two flexible strain sensors is larger than a preset value.
The specific scheme is that the glove comprises a cloth inner layer, a cloth outer layer and an elastic support ring supported between the cloth inner layer and the cloth outer layer; the elastic support ring is composed of a plurality of annular rubber layers which are sleeved and fixedly connected into an integral structure, and the number of the layers is more than five and is an odd number; from inside to outside, the elastic support ring comprises a connecting layer part formed by alternately sleeving a high-elasticity rubber layer and a low-elasticity rubber layer, and a bottom high-elasticity rubber layer for fixedly connecting the inner cloth layer and the connecting layer part, wherein the bottom high-elasticity rubber layer is fixedly connected with the outer surface of the inner cloth layer in an adhering manner, and the elastic rubber layer fixedly connected with the outer cloth layer in an adhering manner is a low-elasticity rubber layer; the elastic support ring is tightly sleeved between the two finger joints and is positioned beside the outer end side finger joint; the wireless chipless flexible sensing component is arranged in the overhead layer structure supported by the elastic support ring and is fixedly connected with the outer surface of the inner layer of the cloth in an adhering mode.
The more specific scheme is that the cloth inlayer is elasticity weaving bed of cloth, and the cloth skin is waterproof weaving bed of cloth, coats on the surface of wireless chipless flexible sensing subassembly and coats and have the protection glue film.
The preferred scheme is that the knuckle sleeve part provided with the wireless chipless sensing assembly is only used for being sleeved outside the knuckle between two knuckles on the finger.
In order to achieve the above further object, the present invention provides a wireless chipless strain sensing assembly including a strain sensor, wherein the strain sensing assembly includes an RFID module having a connection lead connected in parallel with the strain sensor.
The specific scheme is that the strain sensor is a flexible strain gauge.
Drawings
FIG. 1 is a flowchart illustrating a method for recognizing hand gesture in accordance with an embodiment of the present invention;
FIG. 2 is a schematic block diagram of a circuit of a hand gesture recognition system according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a glove and a wireless chipless flexible sensing element and an RFID module disposed thereon according to an embodiment of the present invention;
FIG. 4 is an enlarged view of a portion A of FIG. 3;
FIG. 5 is an exploded view of the upper finger cuff portion of the glove and the detection transmission layer disposed therein in accordance with an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of a wireless chipless sensing assembly in an embodiment of the present invention;
FIG. 7 is a block diagram of a flexible support ring in an embodiment of the present invention;
FIG. 8 is a diagram illustrating filtering of thumb gesture data and dynamic gesture segmentation, in accordance with an embodiment of the present invention;
FIG. 9 is a diagram illustrating filtering of index finger gesture data and dynamic gesture segmentation, according to an embodiment of the present invention;
FIG. 10 is a schematic diagram illustrating filtering of ring finger gesture data and dynamic gesture segmentation in an embodiment of the present invention;
FIG. 11 is a diagram illustrating filtering of little finger gesture data and dynamic gesture segmentation, according to an embodiment of the present disclosure;
FIG. 12 is a diagram illustrating middle finger gesture data filtering and dynamic gesture segmentation according to an embodiment of the present invention;
FIG. 13 is a flowchart of the operation of the identifying step in an embodiment of the present invention.
Detailed Description
The invention is further illustrated by the following examples and figures.
The main idea of the invention is to improve the structure of the sensing component for detection so as to simplify the structure of the wireless chipless sensing detection at the same time.
Examples
Referring to fig. 2, the flexible wearable hand motion gesture recognition system 8 of the present invention includes an upper computer 80 for motion gesture recognition, a glove 1 worn on a monitored object, an RFID read-write module 2 connected to the upper computer 80 in wireless communication, and a wireless chipless sensing assembly 3 disposed on each finger sleeve portion 11 of the glove 1; the glove 1 comprises a palm protective sleeve layer 10 and five finger sleeve parts 11, wherein each finger sleeve part 11 comprises a knuckle sleeve part 12 sleeved outside a proximal knuckle, a knuckle sleeve part 13 sleeved outside a middle knuckle and a knuckle sleeve part 14 sleeved outside a tail-end knuckle; on each finger sleeve part 11, the wireless chipless sensing component 3 is arranged only on the knuckle sleeve part 12 and the knuckle sleeve part 14, that is, in the present embodiment, the knuckle sleeve part on which the wireless chipless sensing component 3 is arranged is only used for being sleeved outside the knuckle between two knuckles on the finger.
The RFID read-write module 2 constitutes the receiving and transmitting end of the embodiment, and is composed of a microprocessor, a memory, a read-write antenna, a power supply and a communication interface, wherein the microprocessor is used for identifying the frequency of the received antenna, analyzing the detection sensing module and the impedance of the antenna, and reversely deducing corresponding dependent variables, namely preprocessing the received radio frequency signal; the microprocessor stores the analysis result into the memory, and when the communication interface receives the reading command, the microprocessor takes out the analysis result and sends the analysis result to the communication interface; the read-write antenna sends a radio frequency signal, receives a feedback radio frequency signal and sends a read value to the microprocessor; the power supply provides power resources for the overall receiving sending end; the communication interface adopts WIFI/bluetooth to connect for accept and reply host computer signal.
In this embodiment, in this example, the data processed by the microprocessor is frequency data in different frequency bands, which eliminates the tedious process that the common RFID needs to rely on the chip ID for identification and then reads the internal data, on one hand, time is saved, efficiency is improved, on the other hand, the processing mode of the microprocessor is simplified, and the applicable range of the microprocessor is expanded.
As shown in fig. 5 and 6, the wireless chipless sensing assembly 3 includes a flexible strain sensor 31 disposed in the knuckle sleeve portion, and an RFID tag module 30 connected in parallel with the flexible strain sensor 31; the flexible strain sensor 31 may be a flexible strain gauge or a piezoelectric sensor made of flexible piezoelectric material, and the resistance between two ends of the signal output changes with the change of the deformation state of the strain wire 32, that is, in this embodiment, the signal output end of the flexible strain sensor 31 is connected to the RFID antenna 30 through a lead 33 and forms a parallel relationship with the antenna. On the same glove 1, the difference of the resistance values between any two flexible strain sensors 31 is larger than a preset value, the frequency change intervals of any two wireless chipless sensing assemblies 3 in the normal use process are not overlapped, so that the frequency collision fed back by all detection transmission modules is avoided, the flexible strain sensors with different resistance values are adopted, the principle that the feedback frequency sections of each detection transmission module are not crossed is implemented, the frequency sent by a receiving and sending end is the same, the data transmission is ensured to be completed by only one receiving and sending end, the signals of the sensors can be distinguished, the signals of 10 flexible strain sensors 31 on the same glove can be received based on the same RFID read-write module 2, and the identification is convenient.
As shown in fig. 2 to 4 and 7, the glove 1 includes a cloth inner layer 50, a cloth outer layer 51, and an elastic support ring 4 supported between the cloth inner layer 50 and the cloth outer layer 51, and cross sections of the three layers form a ring structure which is mutually sleeved, that is, the cloth inner layer 50 is sleeved on the outside of the finger of the object to be measured, the elastic support ring 4 is sleeved on the outside of the cloth inner layer 50, and the cloth outer layer 51 is sleeved on the outside of the elastic support ring 4. Namely, in the embodiment, the glove 1 adopts a three-layer design, and the bottom layer adopts elastic textile fabric, so that the glove is ensured to be closely contacted with the hand, and the measurement inaccuracy caused by relative slippage is reduced; the middle layer is a detection transmission module layer, the detection transmission module is firmly adhered to the bottom cloth by adhering glue by a tabletting method, the detection transmission module is in unilateral contact with the glue during adhesion, the detection transmission module is slowly inclined to be completely adhered, gas is prevented from remaining in the middle to cause unstable adhesion, protective glue is coated on the surface layer of the detection transmission module constructed by the wireless chipless sensing assembly 3, the influence of hand sweat on the detection transmission module caused by long-time wearing of gloves is prevented, and the service life is prolonged; the top layer is made of waterproof textile fabric, so that water is prevented from dripping on the gloves, and damage to the detection and transmission module caused by water flow is avoided; meanwhile, a receiving and sending end is placed at the back of the hand of the top layer. In this embodiment, the glove is designed as a full-size glove, with the top and bottom layers being sewn together at the wrist portion.
As shown in fig. 7, the elastic support ring 4 is formed by a plurality of annular rubber layers which are sleeved and fixedly connected into an integral structure, and the number of the annular rubber layers is more than five and is an odd number; from inside to outside, the elastic support ring 4 comprises a connecting layer part 40 formed by alternately sleeving a high-elasticity rubber layer and a low-elasticity rubber layer, and a bottom high-elasticity rubber layer 41 for fixedly connecting the cloth inner layer 50 and the connecting layer part 40, wherein the bottom high-elasticity rubber layer 41 is fixedly connected with the outer surface of the cloth inner layer 50 in an adhering manner, the elastic rubber layer fixedly connected with the cloth outer layer 41 in an adhering manner is a low-elasticity rubber layer, the number of the annular rubber layer is 5 in the embodiment, namely from inside to outside, and the connecting layer part 40 comprises a high-elasticity rubber layer 42, a low-elasticity rubber layer 43, a high-elasticity rubber layer 44 and a low-elasticity rubber layer 45.
As shown in fig. 3, during the use, the elastic support ring 4 is used to be tightly sleeved between two finger joints and is arranged beside the external end side finger joint, such as in the arrangement of the metacarpophalangeal joint 03 and the middle joint 02 and arranged beside the middle joint 02, and in the arrangement of the end joint 01 and the middle joint 02 and arranged beside the end joint 01; and a mounting gap is reserved between the two elastic support rings 4 and the end parts of the wireless chipless sensing assemblies 3 on the two sides of the two elastic support rings; more than two elastic support rings 7 support an overhead layer structure between the inner layer cloth 50 and the outer layer cloth 51, and are used for accommodating the wireless chipless flexible sensing assembly 3, and the wireless chipless flexible sensing assembly 3 is fixedly connected with the outer surface of the inner 50 layers of the cloth in a bonding manner. In this embodiment, the inner cloth layer 50 is an elastic textile cloth layer, the outer cloth layer 51 is a waterproof textile cloth layer, and a protective adhesive layer is coated on the outer surface of the wireless chipless flexible sensing assembly 3.
As shown in fig. 1, the method for recognizing the hand motion gesture of the object nested in the glove 1 based on the hand motion gesture recognition system 8 includes a transmitting step S1, a receiving step S2 and a recognizing step S3, and the specific processes are as follows:
and a transmitting step S1, wherein the upper computer 80 controls the RFID read-write module 2 to send an inquiry radio frequency signal to the wireless chipless flexible sensing assembly 3 arranged on the glove.
Specifically, the upper computer 80 sends a control instruction to the RFID read-write module 2 through a wireless communication line such as bluetooth and WIFI, that is, the upper computer 80 sends a command for reading the current finger bending strain amount; the RFID read-write module 2 sends an inquiry radio frequency signal through a transmitting antenna according to the control instruction, namely, a receiving and sending end receives a command and sends the command to a microprocessor; the microprocessor controls the read-write antenna to transmit radio frequency signals.
In the receiving step S2, the RFID read/write module 2 receives, through the antenna, the radio frequency signal fed back by the antenna of the RFID tag module 30, where the radio frequency signal carries the current resistance information of the flexible strain sensor 31.
The detection transmission module receives a radio frequency signal to cause electromagnetic induction to transmit a feedback radio frequency signal, and the feedback radio frequency is related to the resistance value of the strain sensor, namely the bending strain of the finger; the read-write antenna receives the feedback frequency, converts the feedback frequency into a digital signal and sends the digital signal to the microprocessor, and the microprocessor analyzes the dependent variable according to the digital signal and stores the dependent variable into the memory; the communication interface sends the stored value to the upper computer.
In the embodiment, the design of the flexible sensor is based on the characteristics of the strain-type flexible sensor and the RFID, and the strain-type flexible sensor can adopt a flexible sensor which changes resistance when a strain gauge, a piezoelectric sensor and the like are deformed, and is connected with the RFID transmission antenna in parallel by reflow soldering. When the resistance of the strain-type flexible sensor changes due to the deformation of the bent finger, the transmission antenna of the RFID also changes the impedance, the feedback frequency of the RFID transmission antenna is closely related to the impedance, the feedback frequency of the RFID transmission antenna changes due to different impedances, and the receiving end reversely deduces the deformation amount of the bent finger according to the difference of the feedback frequency of the RFID. In order to ensure the detection accuracy, in the embodiment, a linear part of the RFID feedback frequency and the impedance change is selected as a working interval of the data glove.
And a recognition step S3, wherein the upper computer 80 outputs a hand motion gesture recognition result by utilizing the trained hand motion gesture recognition network model based on the preprocessed signal data sent by the RFID read-write module 2 received through the wireless communication line.
The recognition step specifically includes a hand modeling step S21, a gesture segmentation step S22, a time warping step S23, and a gesture recognition step S24.
In the hand modeling step S21, the trained recognition model is an appearance model established according to a common hand shape, and for different operators, the hand model needs to be recalibrated before starting operation, so as to ensure that the finger length, joint position, and finger bending amplitude are consistent with those of the current operator.
A gesture division step S22, for each gesture division determination criterion, is based on the velocity and acceleration of each part of the finger. The acquired data is firstly subjected to preliminary filtering and then subsequent operation. Because the acquired data volume is a dependent variable, only the bending change angle can be obtained through conversion of the hand model. Therefore, in order to obtain the angular velocity and the angular acceleration of the finger movement, it is necessary to perform a difference operation using the acquired bending angle. Because the difference operation can cause the accuracy of the data result to be reduced, when the method is applied to a new environment at first, if a new operator is replaced, a transmission module is detected to be replaced, and the like, the threshold value range A used for judging whether the current gesture is completed can be expanded to ensure a larger fault tolerance rate. Fig. 8 to 12 are schematic diagrams illustrating gesture data filtering and dynamic gesture segmentation, wherein black circles represent segmentation points.
A time warping step S23, which performs a classified dynamic time warping process on the divided gestures. The gesture motion appears very random because of different operators and different operating environments. Therefore, firstly, time warping processing is utilized to ensure that the judgment results of operators in all generations are synthesized, wherein the gestures are not recognized currently and are judged, namely gesture templates; the identification is carried out in the same time period, so that the identification accuracy is improved; secondly, because a large amount of key information is lost in actions such as sliding and the like in the gesture actions, the angles and the angular velocities in the time period are sorted in a descending order of priority, the angles and the angular velocities with higher importance degrees are screened and matched with the angles and the angular velocities in the database, and judged gesture data with higher similarity degrees and judgment standards are found.
And a gesture recognition step S24, wherein the gesture recognition matches the data in the regular time period with the judged gesture data with higher similarity one by one, matches the most satisfactory gesture in various gestures, and collects the data into a multilayer neural network algorithm. For the recognized gesture, if the matching degree is higher than a set value, the recognition is considered to be successful, and the threshold value range A is narrowed in the next recognition; if the matching degree is lower than the set value, the recognition fails, and the recognition system is corrected by changing the threshold value range A, the angle, the angular speed importance degree and the like.
According to the hand gesture recognition method, in order to combine the rigid MEMS sensor analysis algorithm with the neural network algorithm, on one hand, the layering basis of the neural network algorithm is increased, and the accuracy is higher; on the other hand, the adjustability of the MEMS sensor algorithm is enhanced, and the use habit of most people is more easily met.
And the upper computer sends a reading signal at intervals and repeats the working process.
In the above embodiment, the upper computer 80 may be a smart phone, a sports bracelet or a computer, and may be connected to a detection system, an entertainment system (VR), a monitoring system (medical treatment), etc. to increase the application range, so that the application cost of the integrated detection and transmission module is low.
In the embodiment, the flexible sensor is combined with the corresponding detection system, and the flexible sensor and the fabric glove are sewn together to realize the design of the data glove. The strain flexible sensor and the RFID transmission module are combined and integrated into the detection transmission module, so that wireless transmission of the strain flexible sensor is realized, equipment replacement and maintenance are greatly simplified, and a lot of limitations on wearing are reduced; meanwhile, the integrated detection transmission module can save detection circuits for detecting the strain flexible sensor, such as a full-bridge detection amplification circuit and a half-bridge detection amplification circuit, so that the complexity of the equipment is greatly reduced. Through placing detection sensing module on the directness between finger joint, can reduce because the crooked inseparable of meeting an emergency class sensor laminating that leads to of joint, detect inaccurate problem. In addition, because the main motion mode of the last knuckle is follow-up, the model establishment is simplified, the detection system is simplified, a detection transmission module is not arranged, and a follow-up model is established to determine the motion state of the last knuckle; of course, the present invention does not exclude the solution of laying the detection sensor module directly on the end.
Claims (5)
1. A hand motion gesture recognition method based on a wearable flexible sensing assembly is characterized by comprising the following steps:
a transmitting step, namely transmitting an inquiry radio frequency signal to a wireless chipless flexible sensing assembly arranged on the glove; the wireless chipless flexible sensing assembly comprises flexible strain sensors arranged in each knuckle sleeve part of the glove and an RFID tag module connected with a lead in parallel with the flexible strain sensors; the knuckle sleeve part provided with the wireless chipless flexible sensing assembly is only used for being sleeved outside a knuckle positioned between two knuckles on a finger; on the same glove, the difference between the resistance values of any two flexible strain sensors is larger than a preset value, and the frequency change intervals of any two wireless chipless flexible sensing assemblies are not overlapped in the using process; the glove comprises a cloth inner layer, a cloth outer layer and an elastic support ring supported between the cloth inner layer and the cloth outer layer; the elastic support ring is composed of a plurality of annular rubber layers which are sleeved and fixedly connected into an integral structure, and the number of the layers is more than five and is an odd number; from inside to outside, the elastic support ring comprises a connecting layer part formed by alternately sleeving a high-elasticity rubber layer and a low-elasticity rubber layer, and a bottom high-elasticity rubber layer for fixedly connecting the inner cloth layer and the connecting layer part, wherein the bottom high-elasticity rubber layer is fixedly connected with the outer surface of the inner cloth layer in an adhering manner, and the elastic rubber layer fixedly connected with the outer cloth layer in an adhering manner is a low-elasticity rubber layer; the elastic support ring is tightly sleeved between the two finger joints and is positioned beside the outer end side finger joint; the wireless chipless flexible sensing assembly is arranged in the overhead layer structure supported by the elastic support ring and is fixedly bonded with the outer surface of the inner layer of the cloth;
receiving a radio frequency signal fed back by an antenna of the RFID label module, wherein the radio frequency signal fed back by the antenna carries current resistance information of the flexible strain sensor;
a step of identification, which is to output a hand action gesture identification result by utilizing a trained hand action gesture identification network model based on the received radio frequency signal; the action state of the end knuckle is the action state drivenly coupled to the middle knuckle.
2. The hand motion gesture recognition method of claim 1, wherein:
the inner layer of the cloth is an elastic textile cloth layer, the outer layer of the cloth is a waterproof textile cloth layer, and a protective adhesive layer is coated on the outer surface of the wireless chipless flexible sensing assembly.
3. A hand motion gesture recognition method according to claim 1 or 2, characterized in that the recognition step comprises:
a hand modeling step S21, wherein the trained recognition model is an appearance model established according to a common hand sample, and for different operators, the hand model needs to be recalibrated before starting operation to ensure that the finger length, the joint position and the finger bending amplitude are consistent with those of the current operator;
a gesture division step S22 of dividing the determination criterion for each gesture based on the velocity and acceleration of each part of the finger; carrying out primary filtering on the acquired data, and then carrying out subsequent operation; in order to obtain the angular velocity and the angular acceleration of the finger movement, the acquired bending angle is required to be used for differential operation; and when the gesture recognition method is applied to the beginning of a new environment every time, a threshold range A for judging whether the current gesture is completed needs to be expanded;
a time warping step S23, wherein the gesture after being divided is classified and dynamically time warped; in the step, the angles and the angular velocities in the same time period are sorted in a descending order of priority, the angles and the angular velocities higher than a certain importance degree are screened and matched with the angles and the angular velocities in the database, and judged gesture data and judgment standards with higher similarity degrees are found out;
a gesture recognition step S24, matching the data in the normalized time period with the judged gesture data with higher similarity one by one, matching the most satisfactory gesture in various gestures, and acquiring the data into a multilayer neural network algorithm; for the recognized gesture, if the matching degree is higher than a set value, the recognition is considered to be successful, and the threshold value range A is narrowed in the next recognition; if the matching degree is lower than the set value, the recognition fails, and the recognition system is corrected by changing the threshold range A and the angle and the importance degree of the angular speed.
4. A flexible wearable hand motion gesture recognition system comprises an upper computer for motion gesture recognition, gloves worn on a monitored object, RFID read-write modules in communication connection with the upper computer, and wireless chipless sensing assemblies arranged on each finger sleeve part of the gloves;
the method is characterized in that:
the wireless chipless sensing assembly comprises a flexible strain sensor and an RFID tag module, wherein the flexible strain sensor is arranged in a knuckle sleeve part of the finger sleeve part, and the RFID tag module is connected with a connecting lead in parallel with the flexible strain sensor; the knuckle sleeve part provided with the wireless chipless sensing assembly is only used for being sleeved outside a knuckle between two knuckles on a finger;
on the same glove, the difference of the resistance values between any two flexible strain sensors is larger than a preset value; the frequency change intervals of any two wireless chipless sensing assemblies in the use process are not overlapped;
the glove comprises a cloth inner layer, a cloth outer layer and an elastic support ring supported between the cloth inner layer and the cloth outer layer;
the elastic support ring is composed of a plurality of annular rubber layers which are sleeved and fixedly connected into an integral structure, and the number of the layers is more than five and is an odd number; from inside to outside, the elastic support ring comprises a connecting layer part formed by alternately sleeving a high-elasticity rubber layer and a low-elasticity rubber layer, and a bottom high-elasticity rubber layer for fixedly connecting the inner cloth layer and the connecting layer part, wherein the bottom high-elasticity rubber layer is fixedly connected with the outer surface of the inner cloth layer in an adhering manner, and the elastic rubber layer fixedly connected with the outer cloth layer in an adhering manner is a low-elasticity rubber layer;
the elastic support ring is tightly sleeved between the two finger joints and is positioned beside the outer end side finger joint; the wireless chipless sensing assembly is arranged in the overhead layer structure supported by the elastic support ring and is fixedly connected with the outer surface of the inner layer of the cloth in an adhering mode.
5. The hand motion gesture recognition system of claim 4, wherein:
the inner layer of the cloth is an elastic textile cloth layer, the outer layer of the cloth is a waterproof textile cloth layer, and a protective adhesive layer is coated on the outer surface of the wireless chipless sensing assembly.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911406527.3A CN111190489B (en) | 2019-12-31 | 2019-12-31 | Flexible wearable hand motion gesture recognition system and method and sensor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911406527.3A CN111190489B (en) | 2019-12-31 | 2019-12-31 | Flexible wearable hand motion gesture recognition system and method and sensor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111190489A CN111190489A (en) | 2020-05-22 |
| CN111190489B true CN111190489B (en) | 2022-02-18 |
Family
ID=70707996
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911406527.3A Active CN111190489B (en) | 2019-12-31 | 2019-12-31 | Flexible wearable hand motion gesture recognition system and method and sensor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111190489B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112097963B (en) * | 2020-08-27 | 2021-10-08 | 西安电子科技大学 | Stress sensor based on chipless radio frequency identification tag |
| US11914093B2 (en) * | 2021-12-07 | 2024-02-27 | Microsoft Technology Licensing, Llc | RF antenna scanning for human movement classification |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101790815A (en) * | 2007-12-10 | 2010-07-28 | 欧姆龙株式会社 | Rfid tag, and system and method for detecting change of rfid tag environment |
| CN105242788A (en) * | 2015-10-29 | 2016-01-13 | 华侨大学 | Bending sensor-based wireless data glove circuit layout and sensor configuration method |
| CN105725312A (en) * | 2014-12-30 | 2016-07-06 | 意法半导体有限公司 | Flexible smart glove |
| CN108255303A (en) * | 2018-01-24 | 2018-07-06 | 重庆邮电大学 | A kind of gesture identification method based on self-control data glove |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3090323B1 (en) * | 2014-01-03 | 2021-07-21 | Pellaton, Eric | Systems and methods for controlling electronic devices using radio frequency identification (rfid) devices |
-
2019
- 2019-12-31 CN CN201911406527.3A patent/CN111190489B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101790815A (en) * | 2007-12-10 | 2010-07-28 | 欧姆龙株式会社 | Rfid tag, and system and method for detecting change of rfid tag environment |
| CN105725312A (en) * | 2014-12-30 | 2016-07-06 | 意法半导体有限公司 | Flexible smart glove |
| CN105242788A (en) * | 2015-10-29 | 2016-01-13 | 华侨大学 | Bending sensor-based wireless data glove circuit layout and sensor configuration method |
| CN108255303A (en) * | 2018-01-24 | 2018-07-06 | 重庆邮电大学 | A kind of gesture identification method based on self-control data glove |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111190489A (en) | 2020-05-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP2021072136A (en) | Methods and devices for combining muscle activity sensor signals and inertial sensor signals for gesture-based control | |
| CN111190489B (en) | Flexible wearable hand motion gesture recognition system and method and sensor | |
| US9880632B2 (en) | Systems, devices, and methods for gesture identification | |
| CN103677265B (en) | Intelligent sensing glove and intelligent sensing method | |
| KR101618301B1 (en) | Wearable device and information input method using the same | |
| US11681369B2 (en) | Control-point activation condition detection for generating corresponding control signals | |
| KR20190092782A (en) | Glove-type Motion Recognizing Apparatus Capable of Recognizing Motion of Fingers and Hands on the Space and Recognizing Method thereof | |
| CN108037821A (en) | A kind of wearable palm band for gesture identification | |
| Pan et al. | State-of-the-art in data gloves: A review of hardware, algorithms, and applications | |
| KR20190092777A (en) | Glove-type Motion Recognizing Apparatus and Recognizing Method thereof | |
| CN113423341A (en) | Method and apparatus for automatic calibration of wearable electrode sensor system | |
| KR101554125B1 (en) | Wearable device and information input method using the same | |
| CN109283998A (en) | Three-dimensional capacitive wearable human-computer interaction device and method | |
| CN111399660A (en) | Gesture recognition equipment and method for sensor | |
| CN107351090A (en) | A kind of robot control system and method | |
| CN208283897U (en) | A kind of wearable device monitoring wrist motion | |
| CN215347222U (en) | Intelligent glove for human-computer interaction | |
| CN114661159A (en) | Glove and hand motion recognition method, device and equipment | |
| CN210166760U (en) | Mouse with pulse and temperature sensing detection structure | |
| CN112099669B (en) | Electret capacitive pressure sensor unit array for wrist back gesture recognition | |
| CN114997227A (en) | Gesture recognition system based on muscle stress | |
| Pan et al. | Review of the State-of-the-Art of Data Gloves | |
| KR20210051277A (en) | Wearable Device for Motion Detecting and Method for Manufacturing Thereof | |
| Cheng et al. | Finger-worn device based hand gesture recognition using long short-term memory | |
| KR20210108201A (en) | A smart gloves for measuring and dating motion of hand |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |