CN111367400A - Gesture recognition device based on flexible sensor - Google Patents
Gesture recognition device based on flexible sensor Download PDFInfo
- Publication number
- CN111367400A CN111367400A CN201811597880.XA CN201811597880A CN111367400A CN 111367400 A CN111367400 A CN 111367400A CN 201811597880 A CN201811597880 A CN 201811597880A CN 111367400 A CN111367400 A CN 111367400A
- Authority
- CN
- China
- Prior art keywords
- gesture
- flexible
- information
- flexible sensor
- gesture recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
Abstract
The application relates to a gesture recognition device based on flexible sensor belongs to human-computer interaction technical field, and the device includes: the gesture detection system comprises a flexible sensor array and a collector connected with the flexible sensor array; the flexible sensor array comprises a plurality of flexible sensors, wherein each flexible sensor comprises first flexible transparent electrodes at two ends, an electrolyte material which is arranged between the first flexible transparent electrodes and has conductivity, and an encapsulation structure for encapsulating the first flexible transparent electrodes and the electrolyte material; the collector is used for collecting the motion signals collected by the flexible sensor; the gesture recognition system is used for acquiring a motion signal; recognizing corresponding gesture information according to the motion signal; the gesture translation system is used for translating the gesture information into corresponding voice information and/or text information to be played; the problem of low communication efficiency between the sign language expression person and other people can be solved; the communication efficiency between the sign language expression person is improved.
Description
Technical Field
The invention relates to a gesture recognition device based on a flexible sensor, and belongs to the technical field of human-computer interaction.
Background
In modern society, effective communication with the deaf-mute has been a difficult problem to solve. Because deaf-mutes have language expression disorders, they need to communicate with others using sign language. However, when the communication target does not understand the sign language, the communication effect is affected, and the communication efficiency is reduced.
Disclosure of Invention
The invention aims to provide a gesture recognition device based on a flexible sensor. In order to achieve the purpose, the invention provides the following technical scheme: the device comprises:
the gesture detection system comprises a flexible sensor array and a collector connected with the flexible sensor array; the flexible sensor array is used for detecting a motion signal generated by a user when the sign language is expressed; the flexible sensor array comprises a plurality of flexible sensors, each flexible sensor comprises first flexible transparent electrodes at two ends, an electrolyte material which is arranged between the first flexible transparent electrodes and has conductivity, and an encapsulation structure for encapsulating the first flexible transparent electrodes and the electrolyte material; the collector is used for collecting the motion signals collected by the flexible sensor and sending the motion signals to the gesture recognition system;
the gesture recognition system is in communication connection with the gesture detection system; the gesture recognition system is used for acquiring the motion signal; recognizing corresponding gesture information according to the motion signal, and sending the gesture information to the gesture translation system;
the gesture translation system is in communication connection with the gesture recognition system; the gesture translation system is used for receiving gesture information sent by the gesture recognition system; and translating the gesture information into corresponding voice information and/or text information for playing.
Optionally, the flexible transducer array comprises at least two sub-arrays; each subarray corresponds to an area of the hand, and the subarrays are used for collecting hand bending signals of the corresponding areas.
Optionally, the at least two sub-arrays include a first sub-array corresponding to a back region of the hand and a second sub-array corresponding to a palm region of the hand, and positions of the flexible transducers in the first sub-array are one-to-one opposite to positions of the flexible transducers in the second sub-array.
Optionally, the at least two sub-arrays include a third sub-array corresponding to the region on the left hand and a fourth sub-array corresponding to the region on the right hand.
Optionally, the flexible sensor array is encapsulated in a flexible glove through an imide film, and the material of the flexible glove and the base of the flexible sensor array is polydimethylsiloxane PDMS material.
Optionally, the two ends of the flexible sensor are provided with second flexible transparent electrodes connected through conductive silver paste, and the second flexible transparent electrodes are used for wirelessly charging the flexible sensor.
Optionally, the gesture recognition system is configured to:
performing gesture tracking on the motion signals, and determining whether corresponding motion signals exist when gestures are expressed;
performing gesture segmentation on the determined motion signals according to different semantics to obtain segmented motion signals;
performing feature extraction on the segmented motion signals to obtain at least one gesture feature signal;
and matching the gesture characteristic signals with gesture templates in a gesture database by using a gesture recognition algorithm to obtain gesture information.
Optionally, the gesture recognition system sends the gesture information to the gesture translation system through a wireless sending module;
and the gesture translation system receives the gesture information sent by the gesture recognition system through a wireless receiving module.
Optionally, the gesture translation system includes an a/D conversion module, a translation module communicatively connected to the a/D conversion module, and a playing module communicatively connected to the translation module;
the A/D conversion module is used for decoding the gesture information to obtain decoded information; sending the decoding information to the translation module;
the translation module is used for translating the decoding information into voice information and/or character information; and sending the voice information and/or the text information to the playing module for playing.
Optionally, the gesture detection system is connected to the gesture recognition system through a microprocessor;
the microprocessor is used for receiving the motion signals acquired by the gesture detection system and carrying out filtering and noise reduction processing on the motion signals; and sending the processed motion signal to the gesture recognition system.
The invention has the beneficial effects that: the gesture detection system comprising the flexible sensor array and the collector connected with the flexible sensor array is arranged to detect the motion signal, send the motion signal to the gesture recognition system for gesture recognition, send the recognized gesture information to the gesture translation system for translation and play, and the problem that the communication efficiency between a sign language presenter and other people is low can be solved; because the sign language can be automatically converted into the voice or the characters, other people can be helped to understand the sign language, and the communication efficiency between the sign language and a sign language expressor is improved.
In addition, the flexible sensor array and the collector are used for collecting motion signals, and the flexible sensor array can be folded and bent to collect hand signals, so that the collected motion signals are more in line with the actual motion conditions of the hands of the user, and the accuracy of the collected motion signals can be improved.
In addition, the wireless charging mode is adopted for wireless charging, so that the problem of complicated use process caused by repeatedly picking up the flexible sensor for charging can be avoided, and the use process can be simplified; in addition, the electrode for charging is also a flexible electrode, so that the flexible sensor array can be still folded and bent when being connected with the electrode, and the normal use of the flexible sensor array is ensured.
The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical solutions of the present invention more clearly understood and to implement them in accordance with the contents of the description, the following detailed description is given with reference to the preferred embodiments of the present invention and the accompanying drawings.
Drawings
FIG. 1 is a schematic structural diagram of a flexible sensor-based gesture recognition apparatus according to an embodiment of the present application;
fig. 2 is a flowchart of a flexible sensor-based gesture recognition apparatus according to another embodiment of the present application.
Detailed Description
The following detailed description of embodiments of the present invention is provided in connection with the accompanying drawings and examples. The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention.
Fig. 1 is a schematic structural diagram of a flexible sensor-based gesture recognition apparatus according to an embodiment of the present application, where as shown in fig. 1, the apparatus at least includes: a gesture detection system 110, a gesture recognition system 120, and a gesture translation system 130.
The gesture detection system 110 includes a flexible sensor array 111 and a collector 112 connected to the flexible sensor array 111.
The flexible sensor array 111 is used to detect motion signals generated by the user when the sign language is expressed. The flexible sensor array 111 includes a plurality of flexible sensors, each of which includes first flexible transparent electrodes at both ends, an electrolyte material disposed between the first flexible transparent electrodes and having conductivity, and an encapsulation structure for encapsulating the first flexible transparent electrodes and the electrolyte material. Illustratively, the flexible sensor is a capacitive sensor, such as: flexible acceleration sensors, etc.
The collector 112 is used for collecting the motion signal collected by the flexible sensor and sending the motion signal to the gesture recognition system.
Optionally, the flexible sensor array comprises at least two sub-arrays; each subarray corresponds to an area of the hand, and the subarrays are used for collecting hand bending signals of the corresponding areas.
Optionally, the at least two sub-arrays include a first sub-array corresponding to the area of the back of the hand and a second sub-array corresponding to the area of the center of the hand, and the positions of the flexible transducers in the first sub-array are opposite to the positions of the flexible transducers in the second sub-array. In this way, the motion signals detected by the first subarray and the motion signals detected by the second subarray may be matched to optimize gesture information to reduce signal acquisition errors.
Optionally, the at least two sub-arrays comprise a third sub-array corresponding to the area on the left hand and a fourth sub-array corresponding to the area on the right hand. In this way, the motion signals detected by the third sub-array and the motion signals detected by the fourth sub-array can be matched with each other, and jointly constitute gesture information.
Optionally, the flexible sensor array is encapsulated in a flexible glove by an imide film, and the material of the flexible glove and the base of the flexible sensor array is Polydimethylsiloxane (PDMS) material. The flexibility and transparency of the PDMS material are good, and the thickness is only dozens of micrometers, so that the wearing comfort of the flexible gloves can be improved. The flexible gloves correspond according to user's finger length, thickness and the thickness of palm and have different models, can avoid causing the great problem of signal error that the sensor gathered because of the size of flexible gloves is unsuitable like this.
Optionally, the flexible gloves are provided with air holes, so that cells of the two hands can breathe normally, and the problems that the flexible gloves are not communicated with outside air to cause palm sweating and shorten the service life of the flexible sensor can be avoided.
Optionally, the two ends of the flexible sensor are provided with second flexible transparent electrodes connected through conductive silver paste, and the second flexible transparent electrodes are used for wirelessly charging the flexible sensor.
Optionally, referring to FIG. 2, the gesture detection system 110 is coupled to the gesture recognition system 120 through a microprocessor 140. The microprocessor 140 is used for receiving the motion signal acquired by the gesture detection system and performing filtering and noise reduction processing on the motion signal; and sending the processed motion signal to a gesture recognition system.
A gesture recognition system 120 communicatively coupled to the gesture detection system 110; the gesture recognition system 120 is used for acquiring a motion signal; and recognizing corresponding gesture information according to the motion signal, and sending the gesture information to the gesture translation system 130.
Illustratively, the gesture recognition system 120 is configured to: performing gesture tracking on the motion signals, and determining whether corresponding motion signals exist when gesture expression exists; performing gesture segmentation on the determined motion signals according to different semantics to obtain segmented motion signals; performing feature extraction on the segmented motion signals to obtain at least one gesture feature signal; and matching the gesture characteristic signals with gesture templates in a gesture database by using a gesture recognition algorithm to obtain gesture information.
Gesture segmentation refers to the process of extracting a meaningful gesture area from an image containing a gesture. For example, the characteristics of semantics, color, gray level and the like which have obvious difference with the meaningless area are extracted. Segmentation methods include, but are not limited to: a segmentation technique based on a contour model; alternatively, the gesture segmentation method is not limited in this embodiment based on the segmentation technique of motion analysis.
Wherein, the gesture feature signals include but are not limited to: the gesture feature signal is not limited in the present embodiment, such as a contour, a position, an area, a distribution, an edge, a distance from a fingertip to a palm center, a histogram feature, and the like.
Optionally, the gesture recognition system 120 may be disposed in a flexible glove along with the gesture detection system 110; alternatively, it may be provided in other devices independent of the gesture detection system 110, such as: the system is arranged in terminals such as a computer, a mobile phone and a tablet personal computer; or in the server, the installation manner of the gesture recognition system 120 is not limited in this embodiment.
Illustratively, referring to fig. 2, the gesture recognition system 120 transmits gesture information to the gesture translation system 130 through the wireless transmission module 121.
Illustratively, referring to fig. 2, the gesture translation system 130 receives the gesture information sent by the gesture recognition system 120 through the wireless receiving module 131.
A gesture translation system 130 communicatively coupled to the gesture recognition system 120; the gesture translation system 130 is to: receiving gesture information sent by the gesture recognition system 120; and translating the gesture information into corresponding voice information and/or text information for playing.
Illustratively, gesture translation system 130 includes an A/D conversion module 132, a translation module 133 communicatively coupled to A/D conversion module 132, and a playback module 134 communicatively coupled to translation module 133.
The a/D conversion module 132 is configured to decode the gesture information to obtain decoded information; send the decoded information to the translation module 133; the translation module 133 is configured to translate the decoded information into voice information and/or text information; and sends the voice message and/or the text message to the playing module 134 for playing.
Optionally, when the translation module 133 translates the voice message, the playing module 134 includes an audio playing module, such as: a speaker; when the translation module 133 translates the text information, the playing module 134 includes a display screen, so as to display the text information.
In summary, in the gesture recognition device based on the flexible sensor provided by this embodiment, the gesture detection system including the flexible sensor array and the collector connected to the flexible sensor array is arranged to detect the motion signal, send the motion signal to the gesture recognition system for gesture recognition, and send the recognized gesture information to the gesture translation system for translation and play, so that the problem of low communication efficiency between the sign language presenter and other people can be solved; because the sign language can be automatically converted into the voice or the characters, other people can be helped to understand the sign language, and the communication efficiency between the sign language and a sign language expressor is improved.
In addition, the flexible sensor array and the collector are used for collecting motion signals, and the flexible sensor array can be folded and bent to collect hand signals, so that the collected motion signals are more in line with the actual motion conditions of the hands of the user, and the accuracy of the collected motion signals can be improved.
In addition, the wireless charging mode is adopted for wireless charging, so that the problem of complicated use process caused by repeatedly picking up the flexible sensor for charging can be avoided, and the use process can be simplified; in addition, the electrode for charging is also a flexible electrode, so that the flexible sensor array can be still folded and bent when being connected with the electrode, and the normal use of the flexible sensor array is ensured.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A flexible sensor based gesture recognition apparatus, the apparatus comprising:
the gesture detection system comprises a flexible sensor array and a collector connected with the flexible sensor array; the flexible sensor array is used for detecting a motion signal generated by a user when the sign language is expressed; the flexible sensor array comprises flexible sensors, each flexible sensor comprises first flexible transparent electrodes at two ends, an electrolyte material which is arranged between the first flexible transparent electrodes and has conductivity, and an encapsulation structure for encapsulating the first flexible transparent electrodes and the electrolyte material; the collector is used for collecting the motion signals collected by the flexible sensor and sending the motion signals to the gesture recognition system;
the gesture recognition system is in communication connection with the gesture detection system; the gesture recognition system is used for acquiring the motion signal; recognizing corresponding gesture information according to the motion signal, and sending the gesture information to the gesture translation system;
the gesture translation system is in communication connection with the gesture recognition system; the gesture translation system is used for receiving gesture information sent by the gesture recognition system; and translating the gesture information into corresponding voice information and/or text information for playing.
2. The apparatus of claim 1, wherein the flexible sensor array comprises at least two sub-arrays; each subarray corresponds to an area of the hand, and the subarrays are used for collecting hand bending signals of the corresponding areas.
3. The apparatus of claim 2, wherein the at least two sub-arrays include a first sub-array corresponding to a dorsal region of the hand and a second sub-array corresponding to a palmar region of the hand, and wherein the flexible transducers in the first sub-array are located one-to-one opposite the flexible transducers in the second sub-array.
4. The apparatus of claim 2, wherein the at least two subarrays comprises a third subarray corresponding to an area on a left hand and a fourth subarray corresponding to an area on a right hand.
5. The device of claim 1, wherein the flexible sensor array is encapsulated in a flexible glove by an imide film, the material of the flexible glove and the base of the flexible sensor array being a Polydimethylsiloxane (PDMS) material.
6. The device according to claim 1, wherein a second flexible transparent electrode connected through a conductive silver paste is disposed at two ends of the flexible sensor, and the second flexible transparent electrode is used for wirelessly charging the flexible sensor.
7. The apparatus of any of claims 1 to 6, wherein the gesture recognition system is configured to:
performing gesture tracking on the motion signals, and determining whether corresponding motion signals exist when gestures are expressed;
performing gesture segmentation on the determined motion signals according to different semantics to obtain segmented motion signals;
performing feature extraction on the segmented motion signals to obtain at least one gesture feature signal;
and matching the gesture characteristic signals with gesture templates in a gesture database by using a gesture recognition algorithm to obtain gesture information.
8. The device according to any one of claims 1 to 6,
the gesture recognition system sends the gesture information to the gesture translation system through a wireless sending module;
and the gesture translation system receives the gesture information sent by the gesture recognition system through a wireless receiving module.
9. The device of any one of claims 1 to 6, wherein the gesture translation system comprises an A/D conversion module, a translation module communicatively coupled to the A/D conversion module, and a playback module communicatively coupled to the translation module;
the A/D conversion module is used for decoding the gesture information to obtain decoded information; sending the decoding information to the translation module;
the translation module is used for translating the decoding information into voice information and/or character information; and sending the voice information and/or the text information to the playing module for playing.
10. The device according to any one of claims 1 to 6, wherein the gesture detection system is connected to the gesture recognition system through a microprocessor;
the microprocessor is used for receiving the motion signals acquired by the gesture detection system and carrying out filtering and noise reduction processing on the motion signals; and sending the processed motion signal to the gesture recognition system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811597880.XA CN111367400A (en) | 2018-12-26 | 2018-12-26 | Gesture recognition device based on flexible sensor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811597880.XA CN111367400A (en) | 2018-12-26 | 2018-12-26 | Gesture recognition device based on flexible sensor |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111367400A true CN111367400A (en) | 2020-07-03 |
Family
ID=71206254
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811597880.XA Pending CN111367400A (en) | 2018-12-26 | 2018-12-26 | Gesture recognition device based on flexible sensor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111367400A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113176825A (en) * | 2021-04-28 | 2021-07-27 | 北京航空航天大学 | Large-area air-isolated gesture recognition system and method |
-
2018
- 2018-12-26 CN CN201811597880.XA patent/CN111367400A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113176825A (en) * | 2021-04-28 | 2021-07-27 | 北京航空航天大学 | Large-area air-isolated gesture recognition system and method |
| CN113176825B (en) * | 2021-04-28 | 2022-07-01 | 北京航空航天大学 | Large-area air-isolated gesture recognition system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10061389B2 (en) | Gesture recognition system and gesture recognition method | |
| US9619024B2 (en) | Virtual input device and virtual input method | |
| EP3646776A1 (en) | Skin checking device, product information determination method, device and system | |
| CN110781881A (en) | Method, device, equipment and storage medium for identifying match scores in video | |
| CN105868519A (en) | Human body characteristic data processing method and apparatus | |
| CN106797672A (en) | Mobile terminal and its control method | |
| Bhardwaj | A Review Paper on Smart Glove-Converts Gestures into Speech and Text | |
| CN112820299A (en) | Voiceprint recognition model training method and device and related equipment | |
| CN108510988A (en) | A kind of speech recognition system and method for deaf-mute | |
| CN111599460A (en) | Telemedicine method and system | |
| CN111367400A (en) | Gesture recognition device based on flexible sensor | |
| CN104980599A (en) | Sign language-voice call method and sign language-voice call system | |
| CN105975220B (en) | Voice printing auxiliary equipment and voice printing system | |
| EP4062824A1 (en) | Electrocardiosignal prediction method and device, terminals, and storage medium | |
| CN109992753A (en) | A kind of translation processing method and terminal device | |
| CN210697629U (en) | Signal acquisition device, mobile terminal and signal analysis system | |
| CN108922224B (en) | Position prompting method and related product | |
| CN110413106A (en) | A kind of augmented reality input method and system based on voice and gesture | |
| CN216772373U (en) | Wearable intelligent glove, recognition component and grip strength detection component | |
| CN111797849A (en) | User activity identification method and device, storage medium and electronic equipment | |
| CN108846817A (en) | Image processing method, device and mobile terminal | |
| CN112883935A (en) | Neural network real-time sign language translation system equipment | |
| CN112099669A (en) | Electret capacitive pressure sensor unit array for wrist back gesture recognition | |
| CN111208907A (en) | Sign language recognition system and method based on electromyographic signals and finger joint deformation signals | |
| CN109754676A (en) | A kind of roller Braille reading pen for the identification of papery books for the blind |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200703 |
|
| WD01 | Invention patent application deemed withdrawn after publication |