CN110624217A - Rehabilitation glove based on multi-sensor fusion and implementation method thereof - Google Patents
Rehabilitation glove based on multi-sensor fusion and implementation method thereof Download PDFInfo
- Publication number
- CN110624217A CN110624217A CN201910899021.4A CN201910899021A CN110624217A CN 110624217 A CN110624217 A CN 110624217A CN 201910899021 A CN201910899021 A CN 201910899021A CN 110624217 A CN110624217 A CN 110624217A
- Authority
- CN
- China
- Prior art keywords
- data
- core control
- sensor
- gesture
- upper computer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B23/00—Exercising apparatus specially adapted for particular parts of the body
- A63B23/035—Exercising apparatus specially adapted for particular parts of the body for limbs, i.e. upper or lower limbs, e.g. simultaneously
- A63B23/12—Exercising apparatus specially adapted for particular parts of the body for limbs, i.e. upper or lower limbs, e.g. simultaneously for upper limbs or related muscles, e.g. chest, upper back or shoulder muscles
- A63B23/16—Exercising apparatus specially adapted for particular parts of the body for limbs, i.e. upper or lower limbs, e.g. simultaneously for upper limbs or related muscles, e.g. chest, upper back or shoulder muscles for hands or fingers
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B24/00—Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
- A63B24/0087—Electric or electronic controls for exercising apparatus of groups A63B21/00 - A63B23/00, e.g. controlling load
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
- A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B2220/00—Measuring of physical parameters relating to sporting activity
- A63B2220/80—Special sensors, transducers or devices therefor
Abstract
The invention discloses a rehabilitation glove based on multi-sensor fusion and an implementation method thereof. The rehabilitation glove is a low-cost hand rehabilitation device based on a motion sensor and a myoelectric sensor, different gestures are recognized by utilizing signals of the motion sensor worn on a finger of a patient and the myoelectric sensor worn on a wrist of the patient, the active motion mode of the patient is adopted, a motion sensing game is matched, and the interestingness of training and the participation degree of the patient are increased. The design has lower cost, and provides more possibility for rehabilitation of cerebral apoplexy in remote areas or areas with poorer medical conditions; the volume is small, data line transmission is not needed, and the carrying and the use are convenient; the energy consumption is low, the heat productivity of the equipment is very small, and the service life of the equipment is prolonged; third-party equipment such as camera assistance is not needed, no special requirements are required for the use scene, and the universality is high; the workload of medical staff is reduced, and the possibility is provided for autonomous rehabilitation; and multiple sensors are fused, so that the identification precision is high, and the real-time identification speed is high.
Description
Technical Field
The invention relates to a hand rehabilitation device, in particular to a rehabilitation glove based on multi-sensor fusion and an implementation method thereof.
Background
At present, most of related postoperative rehabilitation equipment is manually guided by medical workers, the recovery condition is evaluated based on the experience of the medical workers, a reliable quantitative standard is not available, a large number of professional medical workers are needed to participate, and the recovery is difficult to realize in remote areas or areas with poor medical conditions.
The majority of the rehabilitation gloves currently available are based on the following two designs. Firstly, passive form is strong gloves again, pulls through the initiative of gloves device promptly, and the patient is passively followed and is trained, and is very high to the not good and cost of gloves of the repair effect of muscle and nerve. Secondly, gesture recognition is carried out based on the camera, the method needs the assistance of the camera, is troublesome, has relatively high cost, and greatly reduces the accuracy of gesture recognition in use places with weak illumination intensity and high complexity of the background.
Disclosure of Invention
In view of the problems and defects in the prior art, the invention develops a rehabilitation glove based on multi-sensor fusion and an implementation method thereof. The rehabilitation glove is a low-cost hand rehabilitation device based on a motion sensor and a myoelectric sensor, different gestures are recognized by utilizing signals of the motion sensor worn on a finger of a patient and the myoelectric sensor worn on a wrist of the patient, the active motion mode of the patient is adopted, a motion sensing game is matched, and the interestingness of training and the participation degree of the patient are increased. Meanwhile, different training intensity modes are set for patients with different disease conditions, namely the patients need to operate with different force levels to finish training so as to meet the training requirement of the best match. The rehabilitation glove is powered by a battery, wirelessly transmits data through WIFI, and is convenient to carry and use.
The technical scheme adopted by the invention is as follows: a rehabilitation glove based on multi-sensor fusion is characterized by comprising a core control panel fixed on the back of an insulating glove, and a lithium battery and a WIFI transmission module which are respectively connected with the core control panel; the input end of the core control panel is connected with a motion sensor worn on a patient's index finger and a myoelectric sensor worn on a wrist, the output end of the core control panel is connected with a PC upper computer, and the PC upper computer is provided with a gesture acquisition program, a gesture recognition program and a MATLAB-based somatosensory game.
The core control board is provided with an Atmega328p main control chip and a 16-bit ADC (analog-to-digital converter) chip connected with the Atmega328p main control chip, wherein the model of the motion Sensor is MPU6050, the model of the myoelectric Sensor is MyoWareMuscle Sensor, and the model of the WIFI transmission module is ESP 8266; the SCL end of the motion sensor is connected with the SCL end of the core control board, the SDA end is connected with the SDA end of the core control board, the VCC end is connected with 3.3V of the core control board, the ground wire is connected with the ground wire of the core control board, and the INT end is connected with the interrupt signal D2 end of the core control board; a signal VCC end of the electromyographic sensor is connected with the 5V end of the core control board, the ground wire is connected with the ground wire of the core control board, and an output SIG end is connected with an A0 end of the 16-bit ADC analog-to-digital conversion chip; the VCC end of the WIFI transmission module is connected with 3.3V of the core control panel, the GND end of the WIFI transmission module is connected with GND of the core control panel, and the RX end and the TX end of the WIFI transmission module are respectively connected with the RX end and the TX end of the core control panel; the positive pole of lithium cell connects the VIN end of core control panel, and the negative pole of lithium cell connects the GND end of core control panel.
The gesture collection program of the present invention performs the following operations:
starting a PC upper computer, initializing serial port parameters, and setting acquisition times; connecting a PC upper computer with a core control panel arranged on the back of the hand of the insulating glove, waiting for the PC upper computer to send an operation prompt tone, judging whether acquisition times are reached after a user finishes storing sensor data by the PC upper computer through gestures, and returning the program to wait for the PC upper computer to send the operation prompt tone if the acquisition times are not reached; if the acquisition times are reached, displaying waveform data by the PC upper computer; then judging whether the data acquisition is correct or not, if the data acquisition is incorrect, returning the program to the initialization of serial port parameters, and setting the acquisition times; and if the acquisition is correct, storing the data to a PC upper computer, and then, exiting the system by the program.
The gesture recognition program of the present invention performs the following operations: electrifying, starting to initialize serial port parameters, importing trained SVM classifier parameters in the acquired data, and reading sensor data; keeping the static level of the hand, calibrating the data of the sensor, initializing threshold values TH1 and TH2, and starting training after the occurrence of a warning tone with calibration end; reading data frame by frame, comparing the data with a threshold value TH1, judging whether the sensor data is larger than the threshold value TH1, if not, returning the program to the previous step; if yes, judging that the gesture starts, continuously reading fifteen frames of data, then updating threshold values TH1 and TH2, and simultaneously calculating gesture energy; judging whether the energy value is larger than a threshold value TH2, if not, judging to be disturbance, returning the program to read data frame by frame, and comparing with the threshold value TH 1; if yes, sending the data into an SVM classifier for gesture classification, and finally sending the result to a motion sensing game background to execute corresponding operation; judging whether the training is finished or not, if not, returning the program to read data frame by frame and comparing the data with a threshold value TH 1; if so, power is off and the process ends.
A method for realizing rehabilitation gloves based on multi-sensor fusion is characterized by comprising the following steps:
(I) data acquisition
The method comprises the following steps that a summoning volunteer wears a motion sensor and a myoelectric sensor on a rehabilitation glove to perform gesture operation, micro-processing operation is performed through a main control chip on a core control panel, data of a standard unit are output, then a data tag is added to a signal, and the data are transmitted to a PC upper computer through a wifi module; and the same gesture is operated by a plurality of volunteers for a plurality of times and is collected for a plurality of times to obtain a plurality of groups of data information, and finally the data are stored in a gesture database of a PC to be used as training data of the SVM classifier model.
(II) data processing
And (4) performing data preprocessing by adopting a median filter.
(III) model training
And training a classifier model by adopting a Support Vector Machine (SVM) classifier based on a polynomial kernel function and adopting grid search hyper-parameter optimization and cross validation for the collected and processed data set data.
(IV) real-time detection
During real-time detection, the postures of the operator with the palm facing downwards and the flat hand are specified as the start and the end of each gesture, and the posture must be recovered after each movement; and when the data is detected to exceed the threshold value, storing the following continuous 15 frames of data, sending the data into an SVM classifier for recognition, sending the data into a motion sensing game background after a result is obtained, and carrying out corresponding operation in the game.
The beneficial effects produced by the invention are as follows:
(1) the design has lower cost and provides more possibility for rehabilitation of cerebral apoplexy in remote areas or areas with poorer medical conditions.
(2) The volume is small and exquisite, does not need the data line transmission, conveniently carries and uses.
(3) The energy consumption is low, and the calorific capacity of equipment is very little for the life of equipment can prolong.
(4) Third-party equipment such as camera assistance is not needed, special requirements on the use scene are not required, and the universality is high.
(5) Reducing the workload of medical staff and providing possibility for autonomous rehabilitation.
(6) And multiple sensors are fused, so that the identification precision is high, and the real-time identification speed is high.
Drawings
FIG. 1 is a schematic view of the back side of a hand of a rehabilitation glove of the present invention;
FIG. 2 is a schematic view of a palm side of a rehabilitation glove of the present invention;
FIG. 3 is a circuit schematic of the present invention;
FIG. 4 is a flow chart of a gesture data collection host computer of the present invention;
FIG. 5 is a flow chart of a gesture recognition host computer of the present invention;
FIG. 6 is a partial data graph of a gesture database of the present invention;
FIG. 7 is a diagram of an example of a SVM classification result confusion matrix of the present invention;
FIG. 8 is a three-axis schematic view of the motion sensor of the present invention;
FIG. 9 is a schematic diagram of one of the motion sensing game interfaces of the present invention;
FIG. 10 is a schematic diagram of an example of gestures of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
As shown in fig. 1 and 2, the design comprises a core control board 5 fixed on the back of the hand of an insulating glove 2, and a lithium battery 3 and a WIFI transmission module 4 which are respectively connected with the core control board 5; the input end of the core control panel 5 is connected with the motion sensor 1 worn on the index finger of the patient and the myoelectric sensor 6 worn on the wrist of the patient, the output end of the core control panel 5 is connected with the PC upper computer, and the PC upper computer is provided with a gesture acquisition program, a gesture recognition program and a MATLAB-based somatosensory game.
This rehabilitation gloves of design is insulating gloves 2 for preventing static cotton gloves, host computer gesture acquisition program, host computer gesture identification program and the body based on MATLAB research and development feel the recreation and install on the PC. The gesture recognition is realized by utilizing a core control panel which is arranged on the glove body and is designed based on an open source platform. The core control panel is connected with the motion sensor on the forefinger, the myoelectric sensor on the wrist, the WIFI module and the battery respectively, and all signals are transmitted through the WIFI module. In the real-time detection stage, the upper computer gesture recognition program adopts a mode of combining a double threshold value and an SVM classifier, so that the recognition precision is improved.
As shown in fig. 3, the core control board of the design is provided with an Atmega328p main control chip and a 16-bit ADC analog-to-digital conversion chip connected with the Atmega328p main control chip, the model of the motion Sensor is MPU6050, the model of the myoelectric Sensor is MyoWare Muscle Sensor, and the model of the WIFI transmission module is ESP 8266; the SCL end of the motion sensor is connected with the SCL end of the core control board, the SDA end is connected with the SDA end of the core control board, the VCC end is connected with 3.3V of the core control board, the ground wire is connected with the ground wire of the core control board, and the INT end is connected with the interrupt signal D2 end of the core control board; a signal VCC end of the electromyographic sensor is connected with the 5V end of the core control board, the ground wire is connected with the ground wire of the core control board, and an output SIG end is connected with an A0 end of the 16-bit ADC analog-to-digital conversion chip; the VCC end of the WIFI transmission module is connected with 3.3V of the core control panel, the GND end of the WIFI transmission module is connected with GND of the core control panel, and the RX end and the TX end of the WIFI transmission module are respectively connected with the RX end and the TX end of the core control panel; the positive pole of lithium cell connects the VIN end of core control panel, and the negative pole of lithium cell connects the GND end of core control panel.
As shown in fig. 4, the gesture capture program of the present invention performs the following operations:
starting a PC upper computer, initializing serial port parameters, and setting acquisition times; connecting a PC upper computer with a core control panel arranged on the back of the hand of the insulating glove, waiting for the PC upper computer to send an operation prompt tone, judging whether acquisition times are reached after a user finishes storing sensor data by the PC upper computer through gestures, and returning the program to wait for the PC upper computer to send the operation prompt tone if the acquisition times are not reached; if the acquisition times are reached, displaying waveform data by the PC upper computer; then judging whether the data acquisition is correct or not, if the data acquisition is incorrect, returning the program to the initialization of serial port parameters, and setting the acquisition times; and if the acquisition is correct, storing the data to a PC upper computer, and then, exiting the system by the program.
As shown in fig. 5, the gesture recognition program of the present invention performs the following operations: electrifying, starting to initialize serial port parameters, importing a trained SVM classifier model, and reading sensor data; keeping the static level of the hand, calibrating the data of the sensor, initializing threshold values TH1 and TH2, and starting training after the occurrence of a warning tone with calibration end; reading data frame by frame, comparing the data with a threshold value TH1, judging whether the sensor data is larger than the threshold value TH1, if not, returning the program to the previous step; if yes, the gesture is determined to be started, fifteen frames of data are continuously read, then threshold values TH1 and TH2 are updated, and gesture energy E (namely the square sum of the fifteen frames of data) is calculated:(s represents a signal value).
Judging whether the energy value is larger than a threshold value TH2, if not, judging to be disturbance, returning the program to read data frame by frame, and comparing with the threshold value TH 1; if yes, sending the data into an SVM classifier for gesture classification, and finally sending the result to a motion sensing game background to execute corresponding operation; judging whether the training is finished or not, if not, returning the program to read data frame by frame and comparing the data with a threshold value TH 1; if so, power is off and the process ends.
After the WIFI parameters are set at one end of the upper computer, the core control panel is connected with the PC through the WIFI, a user can wear gloves, the hand wearing the motion sensor is kept flat and static, the upper computer reads the sensor parameters to initialize at the moment, the notification program can initialize the double thresholds described above, and the user can start gesture recognition after waiting for the program to send out the prompt tone for successful initialization. The upper computer can scan data streams sent by WIFI continuously, when the upper computer detects that the data signals exceed a signal threshold, the upper computer stores the following continuous 15 frames of signals, at the moment, energy of the fifteen frames of signals is calculated firstly, if the energy is smaller than the energy threshold, disturbance is judged, data are abandoned directly, follow-up analysis is not conducted, and if the energy is larger than the energy threshold, the data are judged to be a group of gesture actions and sent to the classifier for recognition.
After the result is obtained by the classifier, the gesture recognition upper computer transmits the recognition result to the motion sensing game, and different operations in the motion sensing game can be activated by different gesture actions. And displaying on the motion sensing game interface in real time. The user can perform subsequent operations according to the prompt of the motion sensing game and the motion sensing game logic. See fig. 9 for an example of a motion-sensing game operation interface.
The implementation method of the invention comprises the following steps:
(I) data acquisition
The method comprises the following steps that a summoning volunteer wears a motion sensor and a myoelectric sensor on a rehabilitation glove to perform gesture operation, micro-processing operation is performed through a main control chip on a core control panel, data of a standard unit are output, then a data tag is added to a signal, and the data are transmitted to a PC upper computer through a wifi module; and the same gesture is operated by a plurality of volunteers for a plurality of times and is collected for a plurality of times to obtain a plurality of groups of data information, and finally the data are stored in a gesture database of a PC to be used as training data of the SVM classifier model.
The gesture actions are as shown in fig. 10, and in sequence: the index finger clicks, the index finger double clicks, the index finger upwards, the index finger downwards, the index finger leftwards, the index finger rightwards, the index finger and the thumb open and close, the index finger clockwise, the index finger anticlockwise, the palm pulls, the palm pushes, and the fist is held totally twelve gestures.
The device is started and connected with a PC, and an upper computer program in the PC is started. The upper computer program sends out prompt tones at equal intervals, the tester listens the prompt tones and then carries out gesture operation once, and repeats the next gesture movement after hearing the prompt tones again. After the data acquisition of the specified times is completed, the upper computer can observe the data acquisition effect visually (as shown in fig. 6) by the experimenter, and data acquisition errors caused by power failure, instable WIFI connection and other reasons are avoided. And if the experimenter confirms that the data acquisition is correct, storing the data into the PC, and finishing the data acquisition.
Receiving myoelectricity sensor and motion sensor data through the core control panel in this device, received motion sensor data includes: three-axis (X-axis, Y-axis, Z-axis) acceleration data, three-axis angular velocity data (three coordinate axes defined in a rectangular coordinate system, see fig. 8). The gesture database partial data is shown in table 1: the first column is electromyographic sensor data, the columns 2-4 are triaxial acceleration data, the columns 5-7 are angular velocity data, and the column 8 is a label of the number of movements.
The signals are transmitted to an upper computer through a WiFi module or a USB data line. Finally, the data are summarized and stored in a gesture database in the PC machine as training data of the model.
Table 1 gesture database partial data
(II) data processing
And preprocessing the acquired gesture data in a PC (personal computer) by using MATLAB (matrix laboratory) software and adopting a median filter. The method specifically comprises the steps of adopting a sliding time window with a fixed length, sequencing data in the window, and taking an intermediate value as a data value at a midpoint of the time window. The above steps are repeated to process the whole time sequence.
The motion sensor signal has the problems of error and null shift, and in order to improve the precision, the acceleration signal and the angular velocity signal are converted into Euler angles.
(III) model training
And training a classifier model by adopting a SVM classifier based on a polynomial kernel function and adopting grid search hyper-parameter optimization and cross validation. The related hyper-parameters are gamma, degrees and coef, the optimization range of the gamma parameter is 0.001-0.1, the optimization range of the degrees parameter is 2-4, and the optimization range of the coef parameter is 0.001-0.1. The cross validation adopts 10-fold cross validation.
The principle of the SVM classifier is to find the maximum interval of different kinds of data so as to make the classifier have the best reliability. Before the SVM classifier is used for actual classification, the same kind of data is required to be used for training classifier parameters, and a set of all parameters can be referred to as a classifier model for short.
An SVM classifier is a support vector machine, which is a supervised learning mechanism in the field of machine learning, and adopts a kernel function to solve the problem that data is linear inseparable in a low-dimensional space and improve the classification accuracy. An example of the model training results is shown in fig. 7, in which a classifier model is trained using grid search hyper-parameter optimization and cross validation.
(IV) real-time detection
During real-time detection, the postures of the operator with the palm facing downwards and the flat hand are specified as the start and the end of each gesture, and the posture must be recovered after each movement; and when the data is detected to exceed the threshold value, storing the following continuous 15 frames of data, sending the data into an SVM classifier for recognition, sending the data into a motion sensing game background after a result is obtained, and carrying out corresponding operation in the motion sensing game.
During real-time detection, in order to improve the identification precision and avoid misjudgment caused by slight hand disturbance, the SVM classifier is adopted, and signal threshold and energy threshold auxiliary detection is introduced.
Consistent with the above-mentioned actions, the posture of palm-down and hand-flat is still specified as the beginning and end of each gesture, and the gesture must be recovered after each movement.
Claims (5)
1. The rehabilitation gloves based on multi-sensor fusion are characterized by comprising a core control panel (5) fixed on the back of the hand of an insulating glove (2), and a lithium battery (3) and a WIFI transmission module (4) which are respectively connected with the core control panel (5); the input end of the core control panel (5) is connected with the motion sensor (1) worn on the index finger of a patient and the myoelectric sensor (6) worn on the wrist of the patient, the output end of the core control panel (5) is connected with the PC upper computer, and the PC upper computer is provided with a gesture acquisition program, a gesture recognition program and a MATLAB-based somatosensory game.
2. The multi-Sensor fusion-based rehabilitation glove of claim 1, wherein the core control board is provided with an Atmega328p main control chip and a 16-bit ADC (analog-to-digital converter) chip connected with the Atmega328p main control chip, the motion Sensor is MPU6050, the myoelectric Sensor is MyoWare Muscle Sensor, and the WIFI transmission module is ESP 8266; the SCL end of the motion sensor is connected with the SCL end of the core control board, the SDA end is connected with the SDA end of the core control board, the VCC end is connected with 3.3V of the core control board, the ground wire is connected with the ground wire of the core control board, and the INT end is connected with the interrupt signal D2 end of the core control board; a signal VCC end of the electromyographic sensor is connected with the 5V end of the core control board, the ground wire is connected with the ground wire of the core control board, and an output SIG end is connected with an A0 end of the 16-bit ADC analog-to-digital conversion chip; the VCC end of the WIFI transmission module is connected with 3.3V of the core control panel, the GND end of the WIFI transmission module is connected with GND of the core control panel, and the RX end and the TX end of the WIFI transmission module are respectively connected with the RX end and the TX end of the core control panel; the positive pole of lithium cell connects the VIN end of core control panel, and the negative pole of lithium cell connects the GND end of core control panel.
3. The multi-sensor fusion-based rehabilitation glove of claim 1, wherein the gesture collection program performs the following operations:
starting a PC upper computer, initializing serial port parameters, and setting acquisition times; connecting a PC upper computer with a core control panel arranged on the back of the hand of the insulating glove, waiting for the PC upper computer to send an operation prompt tone, judging whether acquisition times are reached after a user finishes storing sensor data by the PC upper computer through gestures, and returning the program to wait for the PC upper computer to send the operation prompt tone if the acquisition times are not reached; if the acquisition times are reached, displaying waveform data by the PC upper computer; then judging whether the data acquisition is correct or not, if the data acquisition is incorrect, returning the program to the initialization of serial port parameters, and setting the acquisition times; and if the acquisition is correct, storing the data to a PC upper computer, and then, exiting the system by the program.
4. The multi-sensor fusion-based rehabilitation glove of claim 1, wherein the gesture recognition program performs the following operations: electrifying, starting to initialize serial port parameters, importing trained SVM classifier parameters in the acquired data, and reading sensor data; keeping the static level of the hand, calibrating the data of the sensor, initializing threshold values TH1 and TH2, and starting training after the occurrence of a warning tone with calibration end; reading data frame by frame, comparing the data with a threshold value TH1, judging whether the sensor data is larger than the threshold value TH1, if not, returning the program to the previous step; if yes, judging that the gesture starts, continuously reading fifteen frames of data, then updating threshold values TH1 and TH2, and simultaneously calculating gesture energy; judging whether the energy value is larger than a threshold value TH2, if not, judging to be disturbance, returning the program to read data frame by frame, and comparing with the threshold value TH 1; if yes, sending the data into an SVM classifier for gesture classification, and finally sending the result to a motion sensing game background to execute corresponding operation; judging whether the training is finished or not, if not, returning the program to read data frame by frame and comparing the data with a threshold value TH 1; if so, power is off and the process ends.
5. A method of performing a rehabilitation operation on a multi-sensor fusion-based rehabilitation glove according to claim 1, characterized in that said method comprises the following steps:
(I) data acquisition
The method comprises the following steps that a summoning volunteer wears a motion sensor and a myoelectric sensor on a rehabilitation glove to perform gesture operation, micro-processing operation is performed through a main control chip on a core control panel, data of a standard unit are output, then a data tag is added to a signal, and the data are transmitted to a PC upper computer through a wifi module; the same gesture is operated by multiple volunteers for multiple times, multiple groups of data information are acquired, and finally the data are stored in a gesture database of a PC (personal computer) and serve as training data of an SVM (support vector machine) classifier model;
(II) data processing
Carrying out data preprocessing by adopting a median filter;
(III) model training
Training a classifier model by adopting grid search hyper-parameter optimization and cross validation for the collected and processed data set data by adopting an SVM classifier based on a polynomial kernel function;
(IV) real-time detection
During real-time detection, the postures of the operator with the palm facing downwards and the flat hand are specified as the start and the end of each gesture, and the posture must be recovered after each movement; and when the data is detected to exceed the threshold value, storing the following continuous 15 frames of data, sending the data into an SVM classifier for recognition, sending the data into a motion sensing game background after a result is obtained, and carrying out corresponding operation in the game.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910899021.4A CN110624217A (en) | 2019-09-23 | 2019-09-23 | Rehabilitation glove based on multi-sensor fusion and implementation method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910899021.4A CN110624217A (en) | 2019-09-23 | 2019-09-23 | Rehabilitation glove based on multi-sensor fusion and implementation method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110624217A true CN110624217A (en) | 2019-12-31 |
Family
ID=68972451
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910899021.4A Pending CN110624217A (en) | 2019-09-23 | 2019-09-23 | Rehabilitation glove based on multi-sensor fusion and implementation method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110624217A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113181619A (en) * | 2021-04-09 | 2021-07-30 | 青岛小鸟看看科技有限公司 | Exercise training method, device and system |
| CN113970968A (en) * | 2021-12-22 | 2022-01-25 | 深圳市心流科技有限公司 | Intelligent bionic hand action pre-judging method |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20110068579A (en) * | 2009-12-16 | 2011-06-22 | 한국전자통신연구원 | Method for finger gesture recognition |
| CN103543826A (en) * | 2013-07-30 | 2014-01-29 | 广东工业大学 | Method for recognizing gesture based on acceleration sensor |
| US20140198034A1 (en) * | 2013-01-14 | 2014-07-17 | Thalmic Labs Inc. | Muscle interface device and method for interacting with content displayed on wearable head mounted displays |
| WO2014130871A1 (en) * | 2013-02-22 | 2014-08-28 | Thalmic Labs Inc. | Methods and devices that combine muscle activity sensor signals and inertial sensor signals for gesture-based control |
| CN105817037A (en) * | 2016-05-19 | 2016-08-03 | 深圳大学 | Toy air vehicle based on myoelectric control and control method thereof |
| CN105929940A (en) * | 2016-04-13 | 2016-09-07 | 哈尔滨工业大学深圳研究生院 | Rapid three-dimensional dynamic gesture recognition method and system based on character value subdivision method |
| CN107728788A (en) * | 2017-10-31 | 2018-02-23 | 吉林大学 | One kind is based on infrared ultrasonic three-dimensional localization body feeling interaction device |
| CN108815804A (en) * | 2018-06-12 | 2018-11-16 | 王晶 | VR rehabilitation training of upper limbs platform and method based on MYO armlet and mobile terminal |
-
2019
- 2019-09-23 CN CN201910899021.4A patent/CN110624217A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20110068579A (en) * | 2009-12-16 | 2011-06-22 | 한국전자통신연구원 | Method for finger gesture recognition |
| US20140198034A1 (en) * | 2013-01-14 | 2014-07-17 | Thalmic Labs Inc. | Muscle interface device and method for interacting with content displayed on wearable head mounted displays |
| WO2014130871A1 (en) * | 2013-02-22 | 2014-08-28 | Thalmic Labs Inc. | Methods and devices that combine muscle activity sensor signals and inertial sensor signals for gesture-based control |
| CN103543826A (en) * | 2013-07-30 | 2014-01-29 | 广东工业大学 | Method for recognizing gesture based on acceleration sensor |
| CN105929940A (en) * | 2016-04-13 | 2016-09-07 | 哈尔滨工业大学深圳研究生院 | Rapid three-dimensional dynamic gesture recognition method and system based on character value subdivision method |
| CN105817037A (en) * | 2016-05-19 | 2016-08-03 | 深圳大学 | Toy air vehicle based on myoelectric control and control method thereof |
| CN107728788A (en) * | 2017-10-31 | 2018-02-23 | 吉林大学 | One kind is based on infrared ultrasonic three-dimensional localization body feeling interaction device |
| CN108815804A (en) * | 2018-06-12 | 2018-11-16 | 王晶 | VR rehabilitation training of upper limbs platform and method based on MYO armlet and mobile terminal |
Non-Patent Citations (1)
| Title |
|---|
| 张发辉，杨大勇，刘婷: "基于肌电信号和姿态信号的手势识别", 《传感器与微系统》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113181619A (en) * | 2021-04-09 | 2021-07-30 | 青岛小鸟看看科技有限公司 | Exercise training method, device and system |
| US11872468B2 (en) | 2021-04-09 | 2024-01-16 | Qingdao Pico Technology Co., Ltd. | Sport training method and system and head-mounted VR device |
| CN113970968A (en) * | 2021-12-22 | 2022-01-25 | 深圳市心流科技有限公司 | Intelligent bionic hand action pre-judging method |
| CN113970968B (en) * | 2021-12-22 | 2022-05-17 | 深圳市心流科技有限公司 | Intelligent bionic hand action pre-judging method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105575219A (en) | Intelligent glove | |
| CN110262664B (en) | Intelligent interactive glove with cognitive ability | |
| JP2021072136A (en) | Methods and devices for combining muscle activity sensor signals and inertial sensor signals for gesture-based control | |
| CN107145236B (en) | Gesture recognition method and system based on wrist tendon pressure related characteristics | |
| CN106174813A (en) | A kind of Intelligent glove | |
| Khodabandelou et al. | Attention-based gated recurrent unit for gesture recognition | |
| CN205508194U (en) | Intelligence gloves | |
| CN110624217A (en) | Rehabilitation glove based on multi-sensor fusion and implementation method thereof | |
| CN102024316B (en) | Wireless intelligent sensing method, device and system | |
| CN206048251U (en) | Gesture identification Apery manipulator system based on Multi-sensor Fusion | |
| CN105014676A (en) | Robot motion control method | |
| CN105919599A (en) | Finger motion detection and identification system and method based on magnetic sensors | |
| CN109613976A (en) | A kind of intelligent flexible pressure sensing hand language recognition device | |
| CN109758154B (en) | Motion state determination method, device, equipment and storage medium | |
| CN113856132B (en) | Rope skipping action recognition system and method and rope skipping | |
| CN106843482A (en) | A kind of Hand gesture detection device based on wireless self-networking pattern | |
| CN116909390A (en) | Multi-mode data acquisition system based on glove | |
| Vishal et al. | Sign language to speech conversion | |
| CN112380943B (en) | Multi-position limb motion capture method based on electrical impedance | |
| Tao et al. | Research on communication APP for deaf and mute people based on face emotion recognition technology | |
| CN114997227A (en) | Gesture recognition system based on muscle stress | |
| CN112883935A (en) | Neural network real-time sign language translation system equipment | |
| CN206101715U (en) | Intelligent glove | |
| CN204748634U (en) | Motion control system of robot | |
| CN110322662B (en) | Posture monitoring and falling alarm bracelet based on physiological and behavioral characteristics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20191231 |