CN110297546B - Device for collecting wrist-finger motion synchronization signals and labeling method thereof - Google Patents
Device for collecting wrist-finger motion synchronization signals and labeling method thereof Download PDFInfo
- Publication number
- CN110297546B CN110297546B CN201910608396.0A CN201910608396A CN110297546B CN 110297546 B CN110297546 B CN 110297546B CN 201910608396 A CN201910608396 A CN 201910608396A CN 110297546 B CN110297546 B CN 110297546B
- Authority
- CN
- China
- Prior art keywords
- finger
- data
- wrist
- microcontroller
- touch
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
Abstract
The invention discloses a device for collecting wrist-finger action synchronous signals and a labeling method thereof, wherein the device is used for collecting synchronous signals when a PVDF wrist action data collecting device (hereinafter referred to as a data collecting device) collects a finger action learning and testing data set. The invention marks the action state (contact/non-contact) data of five fingers corresponding to wrist motion synchronization acquired by the device, and then marks the synchronization signal of the wrist motion training data set acquired by the data acquisition device by using the action state data.
Description
Technical Field
The invention relates to the field of motion recognition, in particular to a device for acquiring wrist finger motion synchronization signals and a labeling method thereof.
Background
Supervised learning is the most common machine learning and is widely applied to the field of motion recognition, training data of the supervised learning is labeled data, and a training target is to be capable of correctly recognizing test data. For example, when gesture motion recognition is performed, some gesture motion data and labels (gesture categories) thereof are trained, and the learning model continuously captures the relation between the gesture motions and the labels to perform self-adjustment and improvement. Then some new gesture actions without labels are input, and the model carries out category identification on the new gesture actions.
Chinese patent No. 201811433180.7: a wearable device based on a PVDF sensor array discloses a wearable device for collecting finger actions, wrist action signals are collected through sensors, machine learning can be carried out based on the wrist action signals, and gesture recognition is achieved.
When the PVDF sensor is used for acquiring a motion data set, the frequently acquired data are signals of continuous motion, and at the moment, how to divide and label the data set becomes a key point, and particularly, when the PVDF sensor is used for labeling the class of the data set, the frequently adopted methods are manual labeling and semi-manual labeling. The manual marking accuracy is high, but the marking workload is large, and a large amount of manpower is consumed; semi-manual labeling generally uses semi-supervised or unsupervised learning to classify data, and then performs manual labeling. Based on the above, it is desirable to have a more convenient method of annotating a data set.
Disclosure of Invention
The invention aims to provide a device for acquiring wrist and finger motion synchronous signals and a labeling method thereof, so as to realize the purpose of labeling data corresponding to wrist motion signals acquired by a PVDF sensor array by using finger state data and solve the problems of data labeling of PVDF sensors in the prior art.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
an apparatus for acquiring a wrist-finger motion synchronization signal, characterized by: the touch control device comprises a touch control panel, a microcontroller and an upper computer, wherein five touch induction sensors corresponding to five fingers are attached to the touch control panel, each touch induction sensor is respectively connected with the microcontroller, and the microcontroller is connected with the upper computer;
each touch induction sensor generates a signal when a finger contacts a touch pad, the signal generated by each touch induction sensor is respectively sent to a microcontroller, the microcontroller converts the signal sent by the touch induction sensor into corresponding data, the data corresponding to the touch induction sensor is used as finger state data of the corresponding finger, the microcontroller counts sampling periods, the count value of each sampling period is used as a finger timestamp, and meanwhile, the microcontroller jointly codes the finger state data and the corresponding timestamp into a data frame;
the microcontroller sends the data frame to the upper computer, the upper computer receives the wrist action signal collected by the data collection device during the finger action and converts the wrist action signal into wrist action data, and the upper computer marks the wrist action data by taking the data frame sent by the microcontroller as a synchronous signal. The data acquisition device in the invention is a wearable device based on a PVDF sensor array, which is disclosed in patent No. 201811433180.7.
The finger timestamp is a count value obtained by progressively counting a counter after data is acquired once in each sampling period (typical value is 1 millisecond), and is also called sampling times.
The device for acquiring the wrist-finger motion synchronous signals is characterized in that: the positions of the five touch sensors on the touch pad can be adjusted according to the positions of the fingers, and the adjusting positions are the same as the arrangement of the finger tip positions of the human hand in a natural state.
The fixing mode of the five touch sensors on the touch plate includes but is not limited to: the back of the touch sensor is attached with pressure-sensitive adhesive which is adhered with the touch pad.
The device for acquiring the wrist-finger motion synchronous signals is characterized in that: the microcontroller respectively stores finger state data of five fingers by using five-bit registers, and binary values 0 and 1 are adopted in each bit register to represent the contact state of the corresponding finger and the touch pad, namely:
when a finger touches a touch pad, a high-level signal generated by a corresponding touch induction sensor is converted into data by a microcontroller and then is stored in a register, and the register assigns a binary value 1 to represent the data;
When a finger does not touch the touch pad, a low-level signal generated by the touch induction sensor is converted into data by the microcontroller and then stored in the register, and the register assigns a binary value of 0 to represent the low-level signal.
The device for acquiring the wrist-finger motion synchronous signals is characterized in that: the microcontroller receives signals of the touch induction sensor in a period, converts the signals into finger state data and stores the finger state data into the register, and meanwhile, the microcontroller uses a counter to progressively count the acquisition times of the finger state data in each sampling period to obtain a count value of the finger state data in each sampling period.
A labeling method of a wrist-finger motion synchronization signal is characterized in that: calculating to obtain a synchronous time reference difference in the upper computer according to the time stamp of the finger state data and the time stamp of the wrist action signal
The calculation formula is as follows:
，
in the above-mentioned formula,
is the system time of the upper computer
At the moment, the timestamp of the first frame wrist action data frame received by the data acquisition device;
Based on synchronous time reference difference
The marking of the corresponding data of the wrist action signal can be realized.
The method for labeling the wrist-finger motion synchronous signal is characterized in that: the finger motion start endpoint may be marked in the wrist motion data using a synchronization-based time reference difference.
The method for labeling the wrist-finger motion synchronous signal is characterized in that: according to the wrist-finger synchronous state characteristic sequence, the finger action type marking can be carried out on the wrist-finger test data.
The invention provides a device for acquiring wrist-finger motion synchronous signals and a labeling method thereof, wherein the device is used for acquiring synchronous signals when a PVDF wrist motion data acquisition device acquires a training data set, and then labeling the training data set by using the synchronous signals, so that the device is a more convenient method for labeling the data set.
Drawings
FIG. 1 is a system schematic of the method of the present invention.
Fig. 2 is a diagram of an application scenario of the apparatus of the present invention.
FIG. 3 is a finger motion initiation endpoint tagging embodiment of the method of the present invention.
In fig. 2: a is a microcontroller unit with data acquisition, control and transmission functions, B1-B5 are touch induction sensors, C is a PVDF wrist action data acquisition device, and D is a touch panel; in fig. 3: e is finger state data collected by B2 when the subject clicks the index finger, and F is a wrist movement signal synchronously collected by the data collection device C.
Detailed Description
The invention is further illustrated with reference to the following figures and examples.
Fig. 1 is a system schematic diagram of a method for labeling a wrist-finger motion synchronization signal according to an embodiment of the present invention, where the method includes:
102, collecting, controlling and transmitting collected finger state data by a microcontroller unit;
103, controlling the time stamp of the synchronous signal by the upper computer;
and 104, marking the training data set.
Wherein the apparatus described in steps 101 and 102 will be described in connection with fig. 2.
In step 103, the timestamp is controlled as follows: for the synchronous acquisition device and the data acquisition device, the synchronous time reference difference is calculated as follows:
. Wherein the content of the first and second substances,
the upper computer is in the system time of the upper computer
A timestamp of a first frame wrist motion data frame received by the time data acquisition device;
the system time of the upper computer is
The time stamp of the first frame of finger synchronization state data frame received by the synchronous acquisition device is disclosed.
In step 104, labeling the training data set includes: the PVDF wrist motion data obtained by the data acquisition device can mark the finger motion starting end point obtained by the synchronous acquisition device.
Furthermore, by comparing the wrist-finger synchronous state characteristic sequences corresponding to the finger action instruction set acquired by the synchronous acquisition device, the finger action type marking can be further carried out on the wrist-finger test data.
As shown in fig. 2, an application scenario diagram of a device for labeling a wrist-finger motion synchronization signal according to an embodiment of the present invention is shown, and the device includes: the touch panel D comprises a touch panel D attached with 5 touch induction sensors B1-B5 and a microcontroller A with data acquisition, control and transmission functions, wherein the number of the touch sensors B1-B5 on the touch panel D is 5, the positions of the touch sensors on the touch panel D can be adjusted according to the positions of fingers of a subject, the adjusting positions are the same as the arrangement of the positions of finger tips of human hands in a natural state, and meanwhile, the touch sensors B1-B5 comprise but are not limited to capacitive touch sensors.
The microcontroller A uses a 5-bit register to store the state data of five fingers in each acquisition period, each bit register represents the state of one finger, and binary values 0 and 1 represent the contact state of the corresponding finger and the touch panel D; further, the microcontroller a uses a counter to count the sampling periods progressively; further, when the finger state data is collected, the collection and storage of the data are completed once per sampling period (the typical value is 1 millisecond); still further, the count value is taken as a time stamp 
And the digital data and the finger state data in the five-bit register are encoded into a data frame, and the data frame is uploaded to an upper computer through a microcontroller unit A and used as a synchronous signal for marking the wrist movement signal synchronously acquired by a data acquisition device C.
As shown in fig. 3, E is finger state data collected by the touch sensor B2 when the subject clicks the index finger, F is one of the channel signals of the wrist movement signal synchronously collected by the data collection device C, and the start point and the end point of the wrist movement signal F synchronously collected by the device C can be detected through E.
When a subject clicks with the thumb, the finger state data acquired by the touch sensors B1-B5 in the wrist-finger motion synchronization signal labeling device is 10000, the finger state data are uploaded to an upper computer, the motion category of the wrist motion signal synchronously acquired by the marking data acquisition device C is thumb clicking, and the motion categories of other fingers are the same.
The wrist signal collecting device C (patent 201811433180.7) encodes the collected wrist motion signal to obtain data in the form of data frames, and the format of the data frames is: the frame header is a timestamp of data in the frame, the timestamp is generated by a counter inside the device, and the frame body is multi-channel data acquired by 10 groups of wrist signal acquisition devices; each group of data comprises a plurality of channels of data at a sampling moment, and the data of the plurality of channels corresponds to the number of sensors of the PVDF sensor array; the format of single channel data at one sampling moment is a binary code with 16 bits, wherein the lower 12 bits are effective data bits; the wrist signal data frame is sent every 10 ms.
The finger action instruction set comprises basic actions such as fingertip clicking, fingertip double clicking, fingertip sliding and the like, and the movement speed of the finger action is within the range: 0.5 actions/second to 10 actions/second, the finger actions include combinations of actions of 5 fingers, such as a simultaneous click of the index finger and the middle finger.
Therefore, the present invention should not be limited to the above-described preferred embodiments, but should be construed to cover all modifications, equivalents, and improvements that fall within the spirit and scope of the present invention.
Claims (5)
1. An apparatus for acquiring a wrist-finger motion synchronization signal, comprising: the touch control device comprises a touch control panel, a microcontroller and an upper computer, wherein five touch induction sensors corresponding to five fingers are attached to the touch control panel, each touch induction sensor is respectively connected with the microcontroller, and the microcontroller is connected with the upper computer;
each touch induction sensor generates a signal when a finger contacts a touch pad, the signal generated by each touch induction sensor is respectively sent to a microcontroller, the microcontroller converts the signal sent by the touch induction sensor into corresponding data, the data corresponding to the touch induction sensor is used as finger state data of the corresponding finger, the microcontroller counts sampling periods, the count value of each sampling period is used as a finger timestamp, and meanwhile, the microcontroller jointly codes the finger state data and the corresponding timestamp into a data frame;
The microcontroller sends the data frames to an upper computer, the upper computer simultaneously receives wrist action signals collected by the data collection device during finger action and converts the wrist action signals into wrist action data, and the upper computer marks the wrist action data by taking the data frames sent by the microcontroller as synchronous signals; in the marking process, calculating to obtain a synchronous time reference difference in the upper computer according to the time stamp of the finger state data and the time stamp of the wrist action signal, and marking a finger movement starting end point in the wrist action data by using the synchronous time reference difference; and marking the finger action type of the wrist-finger test data according to the wrist-finger synchronous state characteristic sequence.
2. A device for acquiring a wrist-finger motion synchronization signal according to claim 1, wherein: the positions of the five touch sensors on the touch pad are adjusted according to the positions of fingers, the adjusting positions are the same as the arrangement of finger tip positions of a human hand in a natural state, and each touch sensor is respectively bonded with the touch pad.
3. A device for acquiring a wrist-finger motion synchronization signal according to claim 1, wherein: the microcontroller respectively stores finger state data of five fingers by using five-bit registers, and binary values 0 and 1 are adopted in each bit register to represent the contact state of the corresponding finger and the touch pad, namely:
When a finger touches the touch pad, a high-level signal generated by the corresponding touch induction sensor is converted into data by the microcontroller and then stored in the register, and the register assigns a binary value 1 to represent the data;
when a finger does not contact the touch pad, a low-level signal generated by the touch induction sensor is converted into data by the microcontroller and then stored in the register, and the register assigns a binary value of 0 to represent the low-level signal.
4. A device for acquiring wrist-finger motion synchronization signals according to claim 1 or 3, wherein: the microcontroller receives signals of the touch induction sensor in a period, converts the signals into finger state data and stores the finger state data into the register, and meanwhile, the microcontroller uses a counter to progressively count the acquisition times of the finger state data in each sampling period to obtain a count value of the finger state data in each sampling period.
5. A labeling method of a wrist-finger motion synchronization signal is characterized in that: calculating to obtain a synchronous time reference difference in the upper computer according to the time stamp of the finger state data and the time stamp of the wrist action signal
The finger state data is obtained through five touch induction sensors corresponding to the five fingers, and the calculation formula is as follows:
In the above-mentioned formula,
is the system time of the upper computer
At the moment, a timestamp of a first frame wrist action data frame received by the data acquisition device, wherein the first frame is a data frame of a first sampling period;
based on synchronous time reference difference
Marking corresponding data of the wrist action signal;
in the marking process, marking a finger motion starting end point in wrist motion data by using the synchronous time reference difference; and marking the finger action type of the wrist-finger test data according to the wrist-finger synchronous state characteristic sequence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910608396.0A CN110297546B (en) | 2019-07-08 | 2019-07-08 | Device for collecting wrist-finger motion synchronization signals and labeling method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910608396.0A CN110297546B (en) | 2019-07-08 | 2019-07-08 | Device for collecting wrist-finger motion synchronization signals and labeling method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110297546A CN110297546A (en) | 2019-10-01 |
| CN110297546B true CN110297546B (en) | 2022-05-03 |
Family
ID=68030628
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910608396.0A Active CN110297546B (en) | 2019-07-08 | 2019-07-08 | Device for collecting wrist-finger motion synchronization signals and labeling method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110297546B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1864199A (en) * | 2003-09-06 | 2006-11-15 | 迈克尔·W·麦克雷 | Hand manipulated data apparatus for computers and video games |
| CN104205015A (en) * | 2012-04-09 | 2014-12-10 | 高通股份有限公司 | Control of remote device based on gestures |
| CN105700673A (en) * | 2014-12-15 | 2016-06-22 | 现代自动车株式会社 | Gesture recognition method in vehicle using wearable device and vehicle for carrying out the same |
| CN108431733A (en) * | 2015-12-31 | 2018-08-21 | 微软技术许可有限责任公司 | Electronic equipment for gestures detection |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017175867A1 (en) * | 2016-04-07 | 2017-10-12 | 国立研究開発法人科学技術振興機構 | Tactile information conversion device, tactile information conversion method, and tactile information conversion program |
-
2019
- 2019-07-08 CN CN201910608396.0A patent/CN110297546B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1864199A (en) * | 2003-09-06 | 2006-11-15 | 迈克尔·W·麦克雷 | Hand manipulated data apparatus for computers and video games |
| CN104205015A (en) * | 2012-04-09 | 2014-12-10 | 高通股份有限公司 | Control of remote device based on gestures |
| CN105700673A (en) * | 2014-12-15 | 2016-06-22 | 现代自动车株式会社 | Gesture recognition method in vehicle using wearable device and vehicle for carrying out the same |
| CN108431733A (en) * | 2015-12-31 | 2018-08-21 | 微软技术许可有限责任公司 | Electronic equipment for gestures detection |
Non-Patent Citations (2)
| Title |
|---|
| Detecting finger gestures with a wrist worn piezoelectric sensor array;Riley Booth;《2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC)》;20171201;全文 * |
| 视频中人的动作分析与理解;刘翠微;《中国优秀博硕士学位论文全文数据库(博士)信息科技辑》;20160415;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110297546A (en) | 2019-10-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2017092225A1 (en) | Emg-based wearable text input system and method | |
| US7719522B2 (en) | Raw data track pad device and system | |
| US20150370333A1 (en) | Systems, devices, and methods for gesture identification | |
| Chung et al. | Real-time hand gesture recognition model using deep learning techniques and EMG signals | |
| CN107145236B (en) | Gesture recognition method and system based on wrist tendon pressure related characteristics | |
| CN1531676A (en) | User input apparatus | |
| CN113849068B (en) | Understanding and interaction method and system for multi-modal information fusion of gestures | |
| CN104856707A (en) | Pressure sensing data glove based on machine vision and gripping process judgment method thereof | |
| CN111481180A (en) | Remote pulse feeling system | |
| Li et al. | Hand gesture recognition and real-time game control based on a wearable band with 6-axis sensors | |
| CN110297546B (en) | Device for collecting wrist-finger motion synchronization signals and labeling method thereof | |
| CN114384999B (en) | User-independent myoelectric gesture recognition system based on self-adaptive learning | |
| Rishikanth et al. | Low-cost intelligent gesture recognition engine for audio-vocally impaired individuals | |
| CN111714121A (en) | Electromyographic data classification model construction method, electromyographic data classification model classification device and server | |
| CN114153317A (en) | Information processing method, device and equipment based on electromyographic signals and storage medium | |
| KR100543710B1 (en) | Spatial information input apparatus and method for recognizing an information-completion signal from a plurality of concurrent spatial motions | |
| CA3188766A1 (en) | Systems and methods decoding intended symbols from neural activity | |
| CN110624217A (en) | Rehabilitation glove based on multi-sensor fusion and implementation method thereof | |
| CN101226439B (en) | Method, system and mouse for simulating flat computer pen to implement hand-written input | |
| CN110362190B (en) | Text input system and method based on MYO | |
| CN206470696U (en) | A kind of identifying device of quick small gesture | |
| CN112380943B (en) | Multi-position limb motion capture method based on electrical impedance | |
| CN104571517A (en) | Body movement based input and control method for electronic device | |
| CN213582081U (en) | Gesture recognition system based on flexible antibacterial biological membrane multi-channel data acquisition module | |
| CN112883935A (en) | Neural network real-time sign language translation system equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230116 Address after: 241000 Building 9, Beihang Navigation Innovation Center, Anhui Xinwu Economic Development Zone, Wanzhou District, Wuhu City, Anhui Province Patentee after: Wuhu Xiliu Photoelectric Technology Co.,Ltd. Address before: 230000 room 902, building 12, union garden, new metropolis, No. 66, Ma'anshan South Road, Baohe District, Hefei City, Anhui Province Patentee before: Hefei Xiliu Photoelectric Technology Co.,Ltd. Effective date of registration: 20230116 Address after: 230000 room 902, building 12, union garden, new metropolis, No. 66, Ma'anshan South Road, Baohe District, Hefei City, Anhui Province Patentee after: Hefei Xiliu Photoelectric Technology Co.,Ltd. Address before: 230009 No. 193, Tunxi Road, Hefei, Anhui Patentee before: Hefei University of Technology |
|
| TR01 | Transfer of patent right |