CN113871028A - Interactive rehabilitation system based on myoelectric intelligent wearing - Google Patents
Interactive rehabilitation system based on myoelectric intelligent wearing Download PDFInfo
- Publication number
- CN113871028A CN113871028A CN202110631860.5A CN202110631860A CN113871028A CN 113871028 A CN113871028 A CN 113871028A CN 202110631860 A CN202110631860 A CN 202110631860A CN 113871028 A CN113871028 A CN 113871028A
- Authority
- CN
- China
- Prior art keywords
- semg
- electromyographic
- signals
- module
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H80/00—ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Abstract
The invention provides an interactive rehabilitation system based on electromyographic intelligent wearing, which comprises wearable electromyographic acquisition equipment, a hand electromyographic active training interface, a physiological state analysis module and a bionic manipulator control module. Wearable myoelectricity collection equipment including: what connect gradually is used for the electrode to paste and lead line, biological electricity signal acquisition module, STM32 microprocessor, WIFI wireless data transmission circuit and gyroscope circuit. The hand myoelectricity active training interface comprises a mobile phone end display screen, a WIFI module and a mobile phone end processor and is used for training hand actions. The physiological state analysis module identifies the current intention action of the user through various methods and sends an action instruction to the bionic manipulator control module. The bionic manipulator control equipment controls the bionic manipulator equipment through wireless transmission according to the result output by the physiological state analysis module. The interaction between the hand motion and the manipulator is achieved through data acquisition, processing and control, and the purpose of hand muscle rehabilitation is further achieved.
Description
Technical Field
The present invention relates to a hand recognition operation system. In particular to an interactive rehabilitation system based on myoelectric intelligent wearing.
Background
In limb movement, hand movement is an indispensable limb language in life, is applied to daily communication in a large number, and with the development of scientific technology, intelligent recognition and application of hand movement gradually enter daily life, and becomes an embodiment of intelligent life. The human hand muscle electric signals are collected, classified identification is carried out, human gesture intentions are understood, relevant responses are made, and the human hand muscle electric signals gradually become an important trend of intelligent life. Before the hand movement, corresponding hand muscle electric signals are generated.
Whenever cells in the hand are stimulated by an electric shock or are activated by nerves, the body generates corresponding muscle potentials. Therefore, the electrical signals of the human muscle are measured at this time, and data analysis is performed, so that the corresponding muscle movement of the human body can be detected and analyzed, or the level of the stimulated neuron can be known. Therefore, when the muscle contracts or expands, the transmitted electric signals can reflect the movement conditions of the nerve and the muscle to a certain extent, and then the training classification is carried out by utilizing an algorithm.
The technology of collecting human body electric signals is a new way to establish information channel between human body and computer. The human body electric signal acquisition technology is used for acquiring and analyzing hand electromyographic signals of a wearer, extracting rich characteristics contained in the electromyographic signals, further judging the hand action state of the wearer, and possibly being used for prosthesis motion control, clinical hand disease detection, clinical diagnosis of motion injury, athlete training, medical rehabilitation and improvement of daily life activities, even in some game and entertainment fields.
However, most of the existing multi-channel myoelectricity acquisition devices have the inconveniences of high price, heavy volume, complex operation, incapability of real-time classification and the like, and the wearable myoelectricity acquisition devices generally have the defects of insufficient precision, fewer channels and the like. Therefore, it is necessary to design and develop a set of high-precision and multi-channel wearable myoelectric acquisition equipment and use the equipment in the field of human body electric signal acquisition.
An embedded single chip microcomputer (STM32) microprocessor with high performance, low cost and low power consumption is widely applied to the application fields of industrial control, consumer electronics, Internet of things, communication equipment, medical service, security monitoring and the like. Wearing formula flesh electricity collection equipment based on STM32 stability and the high performance conceptual design of operation has solved the passageway less, wireless transmission scheduling problem. Therefore, the STM32 is used as a main control chip of the wearable myoelectricity acquisition equipment and matched with the high-precision bioelectricity signal acquisition module, and the acquisition precision of the equipment can be met.
Electromyographic signals are a bioelectrical signal that is generated from any tissue or organ, typically as a function of time and a series of amplitudes, frequencies, and waveforms. The electromyographic signals are bioelectric signals generated along with muscle contraction, and the collected electromyographic signals on the skin surface are called surface electromyographic signals sEMG. sEMG electromyographic signals are bioelectric currents generated by contraction of muscles on the surface of the human body. The nervous system controls the activity of the muscles (contraction or relaxation) and the different muscle fiber motor units in the surface skin generate mutually different signals at the same time. Therefore, the electromyographic signals have the characteristics of nonlinearity, unsteadiness, serious noise interference and the like.
With the rise of deep learning and the improvement of computing power, the real-time judgment of human body movement becomes possible according to the classification judgment of the algorithm. In recent years, deep learning has shown its powerful potential in the fields of object detection, speech recognition, and natural language processing. The deep convolutional neural network is a representative of successful applications in deep learning, and can effectively extract features in the grid-like data. The models of the self-coding neural network and the deep convolution neural network of the multi-layer neurons can exert the corresponding advantages of the respective components, extract the time domain and frequency domain characteristics of the corresponding electromyographic signals, and further realize accurate identification of the hand movements of the wearer.
Disclosure of Invention
The invention aims to solve the technical problem of providing a wearable limb movement intelligent sensing system which can quickly and accurately perform primary identification on the hand movement of a wearer based on myoelectricity so as to control a bionic manipulator.
The technical scheme adopted by the invention is as follows: wearable equipment based on ADS of low-power consumption gathers chip design is used for gathering human surface muscle signal of telecommunication (sEMG), and at the acquisition process, sEMG signal is through filtering, enlargies, and is digital, transmits to the cell-phone end through wireless transmission equipment again, in the dedicated APP of cell-phone, carries out digital filtering, and the application deep learning algorithm trains and classifies. The mobile phone end software platform for developing data analysis is built in the research, various gestures can be classified quickly and accurately, the classification effect can reach a higher level, and real-time waveforms of all channels of sEMG (electromyographic magnetic field) signals can be checked in APP (application) of the mobile phone end. After the signals are collected, the predicted signals can be transmitted to the bionic manipulator equipment to carry out the next action. The higher accuracy rate indicates that the set of system functions is feasible. In the invention, a detailed algorithm structure based on neural network training and a hand recognition action are provided.
Drawings
FIG. 1 is a block diagram of an interactive rehabilitation system based on electromyographic intelligent wearing of the invention;
FIG. 2 is a block diagram of the wearable myoelectricity collecting device according to the present invention;
FIG. 3 is a block diagram of a wireless transmission module according to the present invention;
FIG. 4 is a block diagram of a time-frequency domain feature fusion algorithm according to the present invention;
Detailed Description
The interactive rehabilitation system based on electromyographic intelligent wearing of the invention is described in detail below with reference to examples and drawings. As shown in fig. 1, an interactive rehabilitation system based on myoelectric intelligent wearing comprises: the bionic myoelectricity training device comprises wearable myoelectricity acquisition equipment (1), a hand myoelectricity active training interface (2), a physiological state analysis module (3) and a bionic manipulator module (4). The hand myoelectricity active training device is characterized in that a user collects sEMG (surface EMG) myoelectricity signals from the hand of the user by applying the wearable hand myoelectricity collecting device (1) through the hand myoelectricity active training interface (2).
The physiological state analysis module (3) receives the sEMG electromyographic signals and intelligently identifies hand motions made by a user by combining a deep learning convolutional neural network algorithm (CNN). In the invention, a mobile phone end software platform for developing data analysis is built, namely, the function of the physiological state analysis module (3) can be realized at a mobile phone end, and the waveform of each channel sEMG electromyography acquisition device can be checked. And after intelligent identification is carried out on the mobile phone terminal by utilizing a deep learning convolutional neural network algorithm (CNN), the bionic manipulator equipment (4) is controlled through wireless remote equipment.
The bionic manipulator module (4) takes the physiological state analysis module (3) as a basis, controls the bionic manipulator module (4) to move corresponding to the movement of the hand through wireless remote equipment, and in addition, the gyroscope module sends data through the wireless remote equipment and is combined with the physiological state analysis module (3) to jointly control the bionic manipulator module (4).
As shown in fig. 2, the interactive rehabilitation system worn based on electromyogram intelligence is characterized in that the wearable electromyogram acquisition device (1) comprises: the system comprises electrode patches, a lead wire (11), a bioelectric signal acquisition module (12), an STM32 microprocessor (13), a WIFI wireless data transmission circuit (14), a gyroscope circuit (15) and a wearable electromyography acquisition device (1), wherein the electrode patches and the lead wire are sequentially connected and used for acquiring sEMG electromyography signals, the bioelectric signal acquisition module is used for amplifying and converting the electromyography signals, the STM32 microprocessor (13) is used for controlling the acquisition of the electromyography signals and transmitting the sEMG electromyography signals to the physiological state analysis module (2), and the WIFI wireless data transmission circuit is used for measuring the acceleration, the angular velocity and the angle of the wearable electromyography acquisition device (1); the system comprises a system power supply circuit (16) and a Bluetooth wireless transmission circuit (17), wherein the system power supply circuit (16) is respectively connected with a bioelectric signal acquisition module (12) and an STM32 microprocessor (13), the electrode patches in the electrode patches and the lead wires (11) thereof are used for acquiring sEMG (semG) electromyographic signals of different muscles of hands, and the electrode patches are connected with the bioelectric signal acquisition module (12) through the lead wires and a PJ313B interface and are used for acquiring and transmitting bioelectric signals; the electrode paste is pasted on the forearm of the hand of a wearer, and the muscle measured by the myoelectricity acquisition equipment is as follows: extensor carpi ulnaris, extensor digitorum, extensor radialis brachialis, extensor radialis longus, brachioradialis, circumflex, flexor radialis, flexor palmaris longus, and flexor ulnaris, and myoelectrical signals of 8 channels of the hand can be obtained.
The bioelectrical signal acquisition module (12) consists of a plurality of bioelectrical signal acquisition chips which are integrated with a high common mode rejection ratio analog input AD module for receiving human body surface muscle voltage sEMG signals acquired by the electrode patches, a low-noise programmable gain amplifier for measuring the electromyogram voltage sEMG and amplifying the electromyogram signals and a high-resolution synchronous sampling analog-to-digital converter for converting the analog signals into digital signals;
the STM32 microprocessor (13) is used for adjusting the acquisition mode of the bioelectrical signal acquisition module, adjusting and controlling the WIFI wireless data transmission module (14) and the NRF24L01 wireless remote circuit (17) to output sEMG electromyographic signals, and sending the sEMG electromyographic signals to a mobile phone end so as to be used for the physiological state analysis module (3) to analyze data. And analyzing the acceleration, the angular velocity and the angle signal measured by the gyroscope circuit (15) and finally sending the signals to the bionic manipulator equipment (4).
The WIFI wireless data transmission module (14) works in an AP mode, the highest transmission rate is 4Mbps, and under the control of an STM32 microprocessor (13), collected sEMG electromyographic signals are periodically output to a hand electromyographic active training interface (2) and a physiological state analysis module (3) at a mobile phone end in a data packet mode through the WIFI wireless data transmission module (14).
The Bluetooth wireless transmission circuit (17) is applied to a low-power single-mode module, and the low-power single-mode module is used for a low-power sensor and nearby single-mode equipment. The transmission distance of the Bluetooth wireless transmission circuit (17) reaches more than 100 meters, and the maximum transmitting power can reach 10 dBm; the wireless transmission requirements in the invention can be satisfied. Under the control of an STM32 microprocessor (13), periodically outputting the collected sEMG electromyographic signals to a hand electromyographic active training interface (2) and a physiological state analysis module (3) through a Bluetooth wireless transmission circuit (17) in the form of data packets, and in addition, the Bluetooth wireless transmission circuit (17) can send the data measured by a gyroscope circuit (15) to a bionic manipulator control module (4) after simple analysis of the STM32 microprocessor (13), as shown in FIG. 3;
the gyroscope circuit (15) is mainly applied to a six-axis attitude angle sensor, and a corresponding circuit is designed. The gyroscope circuit (15) adopts a high-precision gyroscope accelerometer MPU6050, reads the measurement data of the MPU6050 through an STM32 microprocessor (13), then outputs through a serial port, integrates an attitude resolver in the module, and can accurately output the acceleration, the angular velocity and the angle which are required to correspond under the dynamic environment by matching with a dynamic Kalman filtering algorithm, the current attitude of the module, the attitude measurement precision is 0.05 degree, and the stability is extremely high. The data output frequency is 100Hz (baud rate 115200) or 20Hz (baud rate 9600), and the requirements on the gyroscope circuit in the invention can be met.
The input voltage of the system power supply circuit (16) is 3.7V, the lithium battery (17) supplies power, and the working voltages of different chips of the system provided by the voltage conversion module are-2.5V, 2.5V and 3.3V respectively.
According to the interactive rehabilitation system based on electromyographic intelligent wearing, a bionic mechanical hand control module (4) is specifically a bionic mechanical palm, five fingers are driven by five anti-rotation LFD-01 steering engines, a tripod head is driven by a 180-degree LD-1501MG digital steering engine, the LD-1501MG digital steering engine is driven to have 17KG torsion, an adapter of a power supply system is a 6V, 5A and DC adapter, the hand of the bionic mechanical palm has 5 degrees of freedom, and a base has 1 degree of freedom.
The interactive rehabilitation system based on electromyographic intelligent wearing is characterized in that the physiological state analysis module (3) has an online real-time classification function, and the real-time online classification function can classify 20 hand actions;
(1) the 20 hand movements are: a vertical thumb gesture, a fist holding gesture, a palm outward swinging gesture, a palm inward swinging gesture, a palm upward swinging gesture, a thumb and forefinger pinching gesture, a thumb and middle finger pinching gesture, a thumb and ring finger pinching gesture, a thumb and little finger pinching gesture, a five-finger tennis ball gripping gesture, a thumb and forefinger water bottle pinching gesture, a cup gripping gesture, a palm cup gripping gesture, a thumb and forefinger pen holding gesture to keep the pen vertical, a palm tennis ball gripping gesture, a brush holding gesture, a pen container holding gesture, a book holding gesture to keep the book vertical, a book holding gesture, and a wooden stick pointing to the front;
firstly, a wearable electromyography acquisition device is used for acquiring sEMG electromyography signals of a wearer, the acquired sEMG electromyography signals are periodically transmitted to a hand electromyography active training interface (2) through a WIFI wireless data transmission module (14), then the actual voltage values of the sEMG electromyography signals are analyzed from A/D conversion results through a conversion algorithm, finally a deep learning Convolutional Neural Network (CNN) model is trained, and after the training is finished, the sEMG electromyography signals are transmitted to a physiological state analysis module (3) through a Bluetooth wireless transmission circuit (17), wherein the actual voltage values of the sEMG electromyography signals are analyzed from A/D conversion results through the conversion algorithm again and are transmitted to the physiological state analysis module (3) for online real-time classification;
the interactive rehabilitation system based on electromyographic intelligent wearing is characterized in that the actual voltage value of the sEMG electromyographic signal is analyzed from the A/D conversion result through a conversion algorithm, and the interactive rehabilitation system based on electromyographic intelligent wearing comprises the following steps:
1) determining a reference voltage V of a bioelectrical signal acquisition module (12)REFAnd the amplification factor G of the programmable gain amplifierPGA；
2) Converting the original A/D conversion result V of each channel16Converted into decimal A/D conversion result V10；
3) Calculating the actual voltage value V of the sEMG electromyographic signal according to the following formulaIN：
Wherein
The interactive rehabilitation system based on electromyographic intelligent wearing is characterized in that the convolutional neural network CNN algorithm based on electromyographic time domain and frequency domain feature fusion comprises the following steps:
1) obtaining raw sEMG electromyographic signals
The method comprises the steps that N is the number of channels of original sEMG electromyographic signals, L is the data length of the original sEMG electromyographic signals of each channel, and represents the g-th numerical value of the original sEMG electromyographic signals collected by the c-th electrode in the original sEMG electromyographic signals;
2) carrying out digital band-pass filtering on the original sEMG electromyographic signals, carrying out 50Hz notch filtering, removing power frequency interference, and obtaining processed sEMG electromyographic signals
Wherein, Xc,gRepresenting the g-th numerical value in the sEMG electromyographic signals corresponding to the c-th electrode after filtering;
3) sEMG electromyographic signal based on digital filtering
After filtering, a plurality of sample sets are constructed.
4) And entering a parameter fine adjustment stage, and sequentially sending the sEMG electromyographic signals after digital filtering of each user into a parallel convolution neural network model with an initial depth for training and gradient correction.
The interactive rehabilitation system based on electromyographic intelligent wearing is characterized in that the step 2) of digitally filtering the original sEMG electromyographic signals adopts a band-pass filter, and the first stop band frequency F of the band-pass filterstop10.001Hz, first passband frequency Fpass110Hz, second pass band frequency Fpass230Hz, second stop band frequency Fstop2The first stopband attenuation rate is 5dB, and the second stopband attenuation percentage is 5dB at 40 Hz.
The interactive rehabilitation system based on electromyographic intelligent wearing is characterized in that the sEMG electromyographic signal sample set obtained after each user is subjected to digital filtering in the step 4) is sequentially sent to a convolutional neural network model with an initial depth. Carrying out full supervision training on the deep convolutional neural network model, setting the initial learning rate of the model to be 0.004, attenuating the learning rate in an exponential form to prevent the fixed learning rate from obtaining the optimal model, carrying out 500-period cyclic training totally, setting the Batchsize to be 128, and setting an early stopping mechanism of Earlystopping. The best model for training the model in all cycles of training is obtained. According to the method, the model parameter fine adjustment is not needed on the basis of pre-training, and when the deep convolutional neural network model is designed, myoelectric data of a plurality of people are collected for training, so that the deep learning model has the cross-tested generalization capability. The output of the deep convolutional neural network model is the result which is output by the physiological state analysis module (2) and corresponds to the motion of the bionic manipulator. Compared with many traditional machine learning algorithms, the CNN uses a multilayer structure to improve the generalization performance and the abstraction performance of the recognition model, and multiple experiments show that the CNN algorithm applied in the invention is an efficient method for hand motion pattern recognition with respect to multi-channel sEMG signal processing.
In the deep convolutional neural network model, forward propagation and backward propagation are mainly utilized. Firstly, analyzing original sEMG electromyographic signal samples through all CNN layers, and forward propagating sEMG electromyographic signal data sets to obtain output values. The error between the output value and the desired value is then calculated to determine the accuracy of the output. Next, the weight values are modified using an error back-propagation process. These two processes are repeatedly performed by the iterative operating system until the loss value of the network is minimized. The weighting values are then modified using a gradient descent algorithm. In order to prevent the final model from being not the optimal model due to excessive modification of the weight, an early stopping mechanism of Earlystopping is arranged in the invention for reducing excessive fitting, and the model with the minimum value of the test loss function is set as the optimal model for training at present.
The deep convolutional neural network model comprises five branches, and the input of each branch is the digital filtered sEMG electromyographic signal; the five branches are different convolution pooling layers, the five convolution pooling layers run in parallel, time domain frequency domain features of the five branches are fused before entering a full-connection layer, the five deep convolution pooling parts respectively emphasize and extract one-dimensional data in a time domain and a frequency domain, the dimensionality of the time domain and the frequency domain features is reduced in a Maxpooling layer, and finally different time domain frequency domain features are output. The five deep convolution pooling parts are deep enough and stable, and can effectively extract the time-frequency domain characteristics of the sEMG electromyographic signals. After the feature fusion, 20 hand motions are classified well;
the first convolutional neural network extracting time domain and frequency domain characteristics sequentially comprises:
(1) the data input layer is used for inputting data which are the sEMG electromyographic signal samples after digital filtering;
(2) the first convolution layer has 64 convolution kernels, the size of which is 20 × 4, and the step size is (1, 1). The convolution kernel selects ReLU as the activation function,
(3) the first maximum pooling layer has the pooling core size of 20 multiplied by 1 and the step length of (1,1), and extracts the maximum value of the elements of the input data covered by the current pooling core as output;
(4) in the second convolutional layer, the number of convolutional kernels is 128, the convolutional kernel size is 3 × 1, and the step size is (1, 1). The convolution kernel selects ReLU as an activation function;
(5) the second maximum pooling layer has the pooling core size of 7 multiplied by 1 and the step length of (1,1) and extracts the maximum value of the elements of the input data covered by the current pooling core as output;
(6) a Flatten layer for one-dimensionalizing the multi-dimensional input;
the second convolutional neural network extracting time domain and frequency domain characteristics sequentially comprises:
(1) the data input layer is used for inputting data which are the sEMG electromyographic signal samples after digital filtering;
(2) the first convolution layer has 64 convolution kernels, the size of which is 16 × 4, and the step size is (1, 1). The convolution kernel selects ReLU as the activation function,
(3) the first maximum pooling layer has the pooling core size of 20 multiplied by 1 and the step length of (1,1), and extracts the maximum value of the elements of the input data covered by the current pooling core as output;
(4) in the second convolutional layer, the number of convolutional kernels is 128, the convolutional kernel size is 4 × 1, and the step size is (1, 1). The convolution kernel selects ReLU as an activation function;
(5) the second maximum pooling layer has the pooling core size of 6 multiplied by 1 and the step length of (1,1) and extracts the maximum value of the elements of the input data covered by the current pooling core as output;
(6) a Flatten layer for one-dimensionalizing the multi-dimensional input;
the third convolutional neural network extracting time domain and frequency domain characteristics sequentially comprises:
(1) the data input layer is used for inputting data which are the sEMG electromyographic signal samples after digital filtering;
(2) the first convolution layer has 64 convolution kernels, the size of 12 × 4 and the step size of (1, 1). The convolution kernel selects ReLU as the activation function,
(3) the first maximum pooling layer has the pooling core size of 20 multiplied by 1 and the step length of (1,1), and extracts the maximum value of the elements of the input data covered by the current pooling core as output;
(4) in the second convolutional layer, the number of convolutional kernels is 128, the convolutional kernel size is 5 × 1, and the step size is (1, 1). The convolution kernel selects ReLU as an activation function;
(5) the second maximum pooling layer, the size of the pooling core is 5 multiplied by 1, the step length is (1,1), and the maximum value of the elements of the input data covered by the current pooling core is extracted and used as output;
(6) a Flatten layer for one-dimensionalizing the multi-dimensional input;
the time domain and frequency domain characteristics extracted by the fourth convolutional neural network sequentially comprise:
(1) the data input layer is used for inputting data which are the sEMG electromyographic signal samples after digital filtering;
(2) the first convolution layer has 64 convolution kernels, 8 × 4 convolution kernels and a step size of (1, 1). The convolution kernel selects ReLU as the activation function,
(3) the first maximum pooling layer has the pooling core size of 20 multiplied by 1 and the step length of (1,1), and extracts the maximum value of the elements of the input data covered by the current pooling core as output;
(4) in the second convolutional layer, the number of convolutional kernels is 128, the convolutional kernel size is 6 × 1, and the step size is (1, 1). The convolution kernel selects ReLU as an activation function;
(5) the second maximum pooling layer, the size of the pooling kernel is 4 multiplied by 1, the step length is (1,1), and the maximum value of the elements of the input data covered by the current pooling kernel is extracted and used as output;
(6) a Flatten layer for one-dimensionalizing the multi-dimensional input;
the time domain and frequency domain characteristics extracted by the fifth convolutional neural network sequentially comprise:
(1) the data input layer is used for inputting data which are the sEMG electromyographic signal samples after digital filtering;
(2) the first convolution layer has 64 convolution kernels, the size of which is 1 × 4, and the step size is (1, 1). The convolution kernel selects ReLU as the activation function,
(3) the first maximum pooling layer has the pooling core size of 20 multiplied by 1 and the step length of (1,1), and extracts the maximum value of the elements of the input data covered by the current pooling core as output;
(4) in the second convolutional layer, the number of convolutional kernels is 128, the convolutional kernel size is 4 × 1, and the step size is (1, 1). The convolution kernel selects ReLU as an activation function;
(5) the second maximum pooling layer, the size of the pooling core is 3 multiplied by 1, the step length is (1,1), and the maximum value of the elements of the input data covered by the current pooling core is extracted and used as output;
(6) a Flatten layer for one-dimensionalizing the multi-dimensional input;
five different time domain and frequency domain characteristics output by the five convolutional neural networks are one-dimensional data, and the data are spliced, namely, the time domain and frequency domain characteristic information is fused, so that the data enter a full connection layer;
(7) the node number of the first full-connection layer is 128, the ReLU is selected as an activation function, the L2 norm is selected as a regularization term, and the L2 norm is set to be 0.004;
(8) the second Dropout layer prevents overfitting from improving the generalization capability of the model, randomly selects the neuron on the upper layer to ensure that the neuron does not output, and the selection probability is 0.3;
(9) the second full-connection layer outputs classification results, the number of nodes is selected as the number of hand actions, Softmax is selected as an activation function, and the Softmax function is a normalized exponential function in nature and is defined as
Wherein e is a natural logarithm value, zhFor the output of the h-th neuron, the denominator in the equation acts as a regularization term, such that
The interactive rehabilitation system based on electromyographic intelligent wearing is characterized in that collected sEMG electromyographic signals of 8 channels are taken as a sample for 250 times before entering a deep learning algorithm, so that the deep learning algorithm can better extract the time domain and frequency domain characteristics of the sEMG electromyographic signals and send the characteristics into the deep learning algorithm.
The wearable limb movement intelligent sensing system based on the electromyography is characterized in that in the sEMG electromyography signals, the electromyography data of each channel can be represented as waveform data of one measured muscle channel. Therefore, in the invention, when the deep learning model is used for semantic segmentation, the conventional spatial convolution pooling of the regular shape NxN is not adopted, but the long and narrow convolution and the strip pooling are adopted. The square sized convolutional and pooling layers limit the correlation and flexibility of upper and lower data information per channel to capture sEMG electromyographic signals in real-time hand movements.
In order to more effectively capture the correlation and flexibility of the sEMG electromyography signals on the upper and lower data information of each channel, the receiving range of a deep learning model is expanded by utilizing long and narrow convolution and stripe pooling, and the correlation characteristics of the upper and lower data of the sEMG electromyography signals are collected. In the convolutional neural network for extracting the first time-domain frequency-domain feature, a long and narrow convolution kernel with the size of 20 × 4 is used to extract the time-domain frequency-domain feature of the sEMG, according to the frequency feature of the sEMG electromyographic signal, the electromyographic frequency is mainly distributed in the range of 0-500Hz, and after digital filtering, the frequency of the sEMG electromyographic signal is completely in the frequency range of 0-500 Hz. The input sEMG electromyographic signals are convoluted by horizontal and vertical long and narrow convolution kernels to calculate H multiplied by 1 and 1 multiplied by W, the H multiplied by 1 and 1 multiplied by W are changed after long and narrow pooling is carried out, the maximum value of the element values in the pooling kernels is obtained, and the maximum value is used as a pooling output value.
In the present invention, in the above mentioned extraction of time domain features in the sEMG electromyogram signal, frequency domain features are extracted to conform to the data waveform features of the sEMG electromyogram signal, and in addition, in the second convolution layer, time domain features of the sEMG electromyogram signal are extracted by long and narrow convolution of 3 × 1 size. Therefore, the time domain and frequency domain characteristics of the sEMG electromyographic signals with different proportions in the first time domain and frequency domain characteristics are extracted, and different prediction results can be obtained after the time domain and frequency domain data with different characteristics are fully connected. The four sEMG electromyographic signals are sequentially repeated for 4 times, time domain and frequency domain characteristics of the four sEMG electromyographic signals are extracted and then sent to a full connection layer, and experiments show that the four sEMG electromyographic signals have a high classification effect.
The interactive rehabilitation system based on electromyographic intelligent wearing is characterized in that the used calculation formula of the long and narrow convolution is as follows:
the interactive rehabilitation system based on electromyographic intelligent wearing is characterized in that in a deep learning model, 20 x 4 long and narrow convolution is applied firstly to extract time domain and frequency domain characteristics of sEMG electromyographic signals, then the time domain and frequency domain characteristics are subjected to maximum value banding pooling of 20 x 1, then the 3 x 1 long and narrow convolution is applied to extract time domain characteristics of sENGl, and the time domain characteristics are subjected to maximum value banding pooling of 7 x 1. In the process of extracting the time-frequency domain characteristics of the first sEMG electromyographic signal, the frequency distribution of each action is subjected to characteristic fusion and dimension reduction for multiple times, and after the stripe pooling layer is applied, the Maxpooling layer has local invariance and can extract obvious characteristics and reduce parameters of the model, so that overfitting of the model is reduced. The Maxpooling only extracts the significant features of the sEMG electromyographic signals and discards insignificant information, the extracted time-frequency domain features can effectively represent the multivariate time sequence of the sEMG electromyographic signals, and the generation of overfitting can be relieved to a certain extent due to the fact that the parameters of the model are reduced. In the invention, a structural block diagram of a deep learning convolutional neural network CNN algorithm of electromyographic signal time domain and frequency domain fusion is shown in FIG. 4.
The invention discloses application of an interactive rehabilitation system based on electromyographic intelligent wearing, and discloses a remote unmanned aerial vehicle control module, which is characterized by comprising the following components in parts by weight:
(1) the result output by the physiological status analysis module (2) of claim 7 is sent to control the bionic manipulator control module (4) through a Bluetooth wireless transmission circuit (17).
The invention discloses an application of an interactive rehabilitation system based on electromyographic intelligent wearing, which is characterized by comprising the following steps:
(1) in order to ensure that the electrode paste is completely contacted with the skin; before measuring the electromyographic signal sEMG, wiping the skin of the measured position of the used hand of the wearer by alcohol cotton, requiring the wearer to keep sitting still, annotating a screen on the eyes and completing the action required by the hand electromyographic active training interface; after a system power supply circuit (16) is ensured to be normal, the system is started, sEMG electromyographic signals are collected through wearable electromyographic collection equipment and are transmitted to a physiological state analysis module, and electromyographic collection work is completed;
(2) the physiological state analysis module automatically carries out digital filtering on the collected sEMG electromyographic signals;
(3) the classification and identification system performs classification and identification according to the digitized filtered sEMG electromyographic signals based on a pre-trained deep convolutional neural network model;
(4) and sending the classification result to a signal control bionic manipulator control module (4) through a Bluetooth wireless transmission circuit (17).
Claims (9)
1. The utility model provides an interactive rehabilitation system based on flesh electricity intelligence is dressed, including connecting gradually: the hand myoelectricity active training device comprises wearable myoelectricity acquisition equipment (1), a hand myoelectricity active training interface (2), a physiological state analysis module (3) and a bionic manipulator control module (4). The hand myoelectricity active training system is characterized in that a user collects sEMG (surface EMG) myoelectricity signals from the hand of the user by applying the wearable hand myoelectricity collecting equipment (1) through the hand myoelectricity active training interface (2); the physiological state analysis module (3) receives the sEMG electromyographic signals and intelligently identifies the hand action selected by the user by combining a deep learning convolutional neural network algorithm (CNN). The bionic manipulator control module (4) takes the physiological state analysis module (3) as a basis, controls the bionic manipulator device (4) to move corresponding to the movement of the hand through wireless remote equipment, and in addition, the gyroscope module also sends data through the wireless remote equipment and is combined with the physiological state analysis module (3) to jointly control the bionic manipulator device (4).
2. The interactive rehabilitation system based on electromyographic intelligent wearing according to claim 1, wherein the wearable electromyographic acquisition device (1) comprises: the electrode paste and the lead wire thereof (11) are connected in sequence and used for collecting sEMG electromyographic signals, a bioelectric signal collecting module (12) used for amplifying and converting the electromyographic signals, an STM32 microprocessor (13) and a WIFI wireless data transmission circuit (14) used for controlling the collection of the electromyographic signals and transmitting the sEMG electromyographic signals to a physiological state analyzing module (2), a gyroscope circuit (15) and a system power supply circuit (16) used for measuring the acceleration, the angular velocity and the angle of the wearable electromyographic signal collecting device (1) and respectively connecting the bioelectric signal collecting module (12) and the STM32 microprocessor (13), and a Bluetooth wireless transmission circuit (17), wherein the electrode paste in the electrode paste and the lead wire thereof (11) collects the sEMG electromyographic signals of different muscles at hands and is connected with the bioelectric signal collecting module (12) through the lead wire and a PJ313B interface, the device is used for collecting and transmitting bioelectricity signals; the electrode paste is pasted on the hand of a wearer, and the muscle collected by the myoelectricity collection module is as follows: extensor carpi ulnaris, extensor digitorum, extensor radialis brevis, extensor radialis longus, brachioradialis, circumflex, flexor radialis, flexor palmaris longus, and flexor radioulnaris, and can obtain the electromyographic signals of 8 channels of the hand;
the bioelectrical signal acquisition module (12) is composed of a plurality of bioelectrical signal acquisition chips which are integrated with an analog input AD module with high common mode rejection ratio for receiving muscle voltage acquired by an electrode patch and sEMG (surface-mounted EMG) electromyographic signals, a low-noise programmable gain amplifier for measuring the electromyographic voltage sEMG and amplifying the electromyographic signals and a high-resolution synchronous sampling analog-to-digital converter for converting the analog signals into digital signals;
the STM32 microprocessor (13) is used for adjusting the acquisition mode of the bioelectrical signal acquisition module, adjusting and controlling the WIFI wireless data transmission module (14) and the NRF24L01 wireless remote circuit (17) to output sEMG electromyographic signals, and sending the sEMG electromyographic signals to a mobile phone end so as to be used for the physiological state analysis module (3) to analyze data. And analyzing the acceleration, the angular velocity and the angle signals measured by the gyroscope circuit (15) and finally sending the signals to the bionic manipulator equipment (4);
the WIFI wireless data transmission module (14) works in an AP mode, the highest transmission rate is 4Mbps, and under the control of an STM32 microprocessor (13), collected sEMG electromyographic signals are periodically output to a hand electromyographic active training interface (2) and a physiological state analysis module (3) at a mobile phone end in the form of data packets through the WIFI wireless data transmission module (14);
the WIFI wireless data transmission module (14) works in an AP mode, the highest transmission rate is 4Mbps, and under the control of an STM32 microprocessor (13), collected sEMG electromyographic signals are periodically output to a hand electromyographic active training interface (2) and a physiological state analysis module (3) through the WIFI wireless data transmission module (14) in the form of data packets;
the Bluetooth wireless transmission circuit (17) is applied to a low-power single-mode module, and the low-power single-mode module is used for a low-power sensor and nearby single-mode equipment. The transmission distance of the Bluetooth wireless transmission circuit (17) reaches more than 100 meters, and the maximum transmitting power can reach 10 dBm; the wireless transmission requirements in the invention can be satisfied. Under the control of an STM32 microprocessor (13), periodically outputting the collected sEMG electromyographic signals to a hand electromyographic active training interface (2) and a physiological state analysis module (3) through a Bluetooth wireless transmission circuit (17) in the form of data packets, and in addition, the Bluetooth wireless transmission circuit (17) can send the data measured by a gyroscope circuit (15) to a bionic manipulator control module (4) after simple analysis of the STM32 microprocessor (13);
the gyroscope circuit (15) is mainly applied to a six-axis attitude angle sensor, and a corresponding circuit is designed. The gyroscope circuit (15) adopts a high-precision gyroscope accelerometer MPU6050, reads the measurement data of the MPU6050 through an STM32 microprocessor (13), then outputs through a serial port, integrates an attitude resolver in the module, and can accurately output the acceleration, the angular velocity and the angle which are required to correspond under the dynamic environment by matching with a dynamic Kalman filtering algorithm, the current attitude of the module, the attitude measurement precision is 0.05 degree, and the stability is extremely high. The data output frequency is 100Hz (baud rate 115200) or 20Hz (baud rate 9600), and the requirements on the gyroscope circuit in the invention can be met;
the input voltage of the system power supply circuit (16) is 3.7V, the system power supply circuit is powered by a lithium battery (17), and the working voltages of different chips of the system, which are provided by the voltage conversion module, are-2.5V, 2.5V and 3.3V respectively;
according to the interactive rehabilitation system based on electromyographic intelligent wearing, a bionic mechanical hand control module (4) is specifically a bionic mechanical palm, five fingers are driven by five anti-rotation LFD-01 steering engines, a tripod head is driven by a 180-degree LD-1501MG digital steering engine, the LD-1501MG digital steering engine is driven to have 17KG torsion, an adapter of a power supply system is a 6V, 5A and DC adapter, the hand of the bionic mechanical palm has 5 degrees of freedom, and a base has 1 degree of freedom.
3. The interactive rehabilitation system based on electromyographic intelligence wearing according to claim 1, wherein the physiological status analysis module (3) has an online real-time classification function, and the real-time online classification mode comprises 20 hand movements;
the 20 hand movements are: a vertical thumb gesture, a fist holding gesture, a palm outward swinging gesture, a palm inward swinging gesture, a palm upward swinging gesture, a thumb and forefinger pinching gesture, a thumb and middle finger pinching gesture, a thumb and ring finger pinching gesture, a thumb and little finger pinching gesture, a five-finger tennis ball gripping gesture, a thumb and forefinger water bottle pinching gesture, a cup gripping gesture, a palm cup gripping gesture, a thumb and forefinger pen holding gesture to keep the pen vertical, a palm tennis ball gripping gesture, a brush holding gesture, a pen container holding gesture, a book holding gesture to keep the book vertical, a book holding gesture, and a wooden stick pointing to the front;
firstly, sEMG electromyographic signals of a wearer are collected through wearable electromyographic collecting equipment, the collected sEMG electromyographic signals are periodically transmitted to a hand electromyographic active training interface (2) through a WIFI wireless data transmission module (14) and a Bluetooth wireless transmission circuit (17), then actual voltage values of the sEMG electromyographic signals are analyzed from A/D conversion results through a conversion algorithm, finally a deep learning Convolutional Neural Network (CNN) model is trained, after training is completed, the sEMG electromyographic signals are transmitted to a physiological state analysis module (3) through the Bluetooth wireless transmission circuit (17), wherein the actual voltage values of the sEMG electromyographic signals are analyzed from the A/D conversion results through the conversion algorithm again, and the actual voltage values are sent to the physiological state analysis module (3) for online real-time classification.
4. The interactive rehabilitation system based on electromyographic intelligent wearing according to claim 3, wherein the actual voltage value of the sEMG electromyographic signal is analyzed from the A/D conversion result through a conversion algorithm, comprising the following steps:
1) determining a reference voltage V of a bioelectrical signal acquisition module (12)REFAnd the amplification factor G of the programmable gain amplifierPGA；
2) Converting the original A/D conversion result V of each channel16Converted into decimal A/D conversion result V10；
3) Calculating the actual voltage value V of the sEMG electromyographic signal according to the following formulaIN：
Wherein
5. The interactive rehabilitation system based on electromyographic intelligent wearing according to claim 3, wherein the parallel CNN algorithm based on the electromyographic time domain frequency domain feature fusion convolutional neural network comprises the following steps:
1) obtaining raw sEMG electromyographic signals
Wherein N is the number of channels of the original sEMG electromyographic signals, L is the data length of the original sEMG electromyographic signals of each channel, and Ec,gThe g-th numerical value in the original sEMG electromyographic signals collected by the c-th electrode in the original sEMG electromyographic signals is represented;
2) carrying out digital band-pass filtering on the original sEMG electromyographic signals, carrying out 50Hz notch filtering, removing power frequency interference, and obtaining processed sEMG electromyographic signals
Wherein, Xc,gRepresents the c-th electrode pair after filteringThe g-th numerical value in the corresponding sEMG electromyographic signals;
3) sEMG electromyographic signal based on digital filtering
Constructing a plurality of sample sets after filtering;
4) and entering a parameter fine adjustment stage, and sequentially sending the sEMG electromyographic signals after digital filtering of each user into a convolution neural network model with an initial depth for training and gradient correction.
6. The interactive rehabilitation system based on electromyographic intelligent wearing according to claim 5, wherein the step 2) of digitally bandpass filtering the raw sEMG electromyographic signals employs a Chebyshev I-type bandpass filter, and a first stop band frequency F of the bandpass filter isstop10.001Hz, first passband frequency Fpass110Hz, second pass band frequency Fpass230Hz, second stop band frequency Fstop2The first stopband attenuation rate is 5dB, and the second stopband attenuation percentage is 5dB at 40 Hz.
7. The interactive rehabilitation system based on electromyographic intelligence wearing according to claim 5, wherein the digitally filtered sEMG electromyographic signal sample sets of each user of step 4) are sequentially fed into the convolutional neural network model at an initial depth. The deep convolutional neural network model is subjected to full supervision training by using TensorFlow, the initial learning rate of the model is set to be 0.004, the learning rate is attenuated in an exponential form so as to prevent the fixed learning rate from failing to obtain the optimal model, 500 cycles of cyclic training are carried out totally, the Batchsize is 128, and an early stopping mechanism of Earlystopping is set. The best model for training the model in all cycles of training is obtained. According to the method, the model parameter fine adjustment is not needed on the basis of pre-training, and when the deep convolutional neural network model is designed, myoelectric data of a plurality of people are collected for training, so that the deep learning model has the cross-tested generalization capability. And the output of the deep convolutional neural network model is a result which is output by the physiological state analysis module (2) and corresponds to the motion of the unmanned aerial vehicle. Compared with many traditional machine learning algorithms, the CNN uses a multilayer structure to improve the generalization performance and the abstract performance of the recognition model, and multiple experiments show that the CNN algorithm applied in the invention is an efficient method for recognizing hand motion modes with respect to multi-channel sEMG signal processing;
in the deep convolutional neural network model, forward propagation and backward propagation are mainly utilized. Firstly, analyzing original sEMG electromyographic signal samples through all CNN layers, and forward propagating sEMG electromyographic signal data sets to obtain output values. The error between the output value and the desired value is then calculated to determine the accuracy of the output. Next, the weight values are modified using an error back-propagation process. These two processes are repeatedly performed by the iterative operating system until the loss value of the network is minimized. The weighting values are then modified using a gradient descent algorithm. In order to prevent the final model from being not the optimal model due to excessive modification of the weight, an early stopping mechanism of Earlystopping is set in the invention, and the model with the minimum value of the test loss function is set as the optimal model for the current training;
the deep convolutional neural network model comprises four branches, and the input of each branch is the digital filtered sEMG electromyographic signal; the four branches are different convolution pooling layers, the four convolution pooling layers run in parallel, time domain and frequency domain characteristics of the four branches are fused before entering a full connection layer, and classification of 20 hand actions is well classified after the characteristic fusion is carried out;
the first convolutional neural network extracting time domain and frequency domain characteristics sequentially comprises:
(1) the data input layer is used for inputting data which are the sEMG electromyographic signal samples after digital filtering;
(2) the first convolution layer has 64 convolution kernels, the size of which is 20 × 4, and the step size is (1, 1). The convolution kernel selects ReLU as the activation function,
(3) the first maximum pooling layer has the pooling core size of 20 multiplied by 1 and the step length of (1,1), and extracts the maximum value of the elements of the input data covered by the current pooling core as output;
(4) in the second convolutional layer, the number of convolutional kernels is 128, the convolutional kernel size is 3 × 1, and the step size is (1, 1). The convolution kernel selects ReLU as an activation function;
(5) the second maximum pooling layer has the pooling core size of 7 multiplied by 1 and the step length of (1,1) and extracts the maximum value of the elements of the input data covered by the current pooling core as output;
(6) a Flatten layer for one-dimensionalizing the multi-dimensional input;
the second convolutional neural network extracting time domain and frequency domain characteristics sequentially comprises:
(1) the data input layer is used for inputting data which are the sEMG electromyographic signal samples after digital filtering;
(2) the first convolution layer has 64 convolution kernels, the size of which is 16 × 4, and the step size is (1, 1). The convolution kernel selects ReLU as the activation function,
(3) the first maximum pooling layer has the pooling core size of 20 multiplied by 1 and the step length of (1,1), and extracts the maximum value of the elements of the input data covered by the current pooling core as output;
(4) in the second convolutional layer, the number of convolutional kernels is 128, the convolutional kernel size is 4 × 1, and the step size is (1, 1). The convolution kernel selects ReLU as an activation function;
(5) the second maximum pooling layer has the pooling core size of 6 multiplied by 1 and the step length of (1,1) and extracts the maximum value of the elements of the input data covered by the current pooling core as output;
(6) a Flatten layer for one-dimensionalizing the multi-dimensional input;
the third convolutional neural network extracting time domain and frequency domain characteristics sequentially comprises:
(1) the data input layer is used for inputting data which are the sEMG electromyographic signal samples after digital filtering;
(2) the first convolution layer has 64 convolution kernels and size12 × 4, step size (1, 1). The convolution kernel selects ReLU as the activation function,
(3) the first maximum pooling layer has the pooling core size of 20 multiplied by 1 and the step length of (1,1), and extracts the maximum value of the elements of the input data covered by the current pooling core as output;
(4) in the second convolutional layer, the number of convolutional kernels is 128, the convolutional kernel size is 5 × 1, and the step size is (1, 1). The convolution kernel selects ReLU as an activation function;
(5) the second maximum pooling layer, the size of the pooling core is 5 multiplied by 1, the step length is (1,1), and the maximum value of the elements of the input data covered by the current pooling core is extracted and used as output;
(6) a Flatten layer for one-dimensionalizing the multi-dimensional input;
the time domain and frequency domain characteristics extracted by the fourth convolutional neural network sequentially comprise:
(1) the data input layer is used for inputting data which are the sEMG electromyographic signal samples after digital filtering;
(2) the first convolution layer has 64 convolution kernels, 8 × 4 convolution kernels and a step size of (1, 1). The convolution kernel selects ReLU as the activation function,
(3) the first maximum pooling layer has the pooling core size of 20 multiplied by 1 and the step length of (1,1), and extracts the maximum value of the elements of the input data covered by the current pooling core as output;
(4) in the second convolutional layer, the number of convolutional kernels is 128, the convolutional kernel size is 6 × 1, and the step size is (1, 1). The convolution kernel selects ReLU as an activation function;
(5) the second maximum pooling layer, the size of the pooling kernel is 4 multiplied by 1, the step length is (1,1), and the maximum value of the elements of the input data covered by the current pooling kernel is extracted and used as output;
(6) a Flatten layer for one-dimensionalizing the multi-dimensional input;
four different time domain and frequency domain characteristics output by the four convolutional neural networks are one-dimensional data, and the data are spliced, namely, the time domain and frequency domain characteristic information is fused, so that the data enter a full connection layer;
(7) the node number of the first full-connection layer is 128, the ReLU is selected as an activation function, the L2 norm is selected as a regularization term, and the L2 norm is set to be 0.004;
(8) the second Dropout layer prevents overfitting from improving the generalization capability of the model, randomly selects the neuron on the upper layer to ensure that the neuron does not output, and the selection probability is 0.3;
(9) the second full-connection layer outputs classification results, the number of nodes is selected as the number of hand actions, Softmax is selected as an activation function, and the Softmax function is a normalized exponential function in nature and is defined as
Wherein e is a natural logarithm value, zhFor the output of the h-th neuron, the denominator in the equation acts as a regularization term, such that
The interactive rehabilitation system based on electromyographic intelligent wearing is characterized in that collected sEMG electromyographic signals of 8 channels are taken as a sample 250 times before entering a deep learning algorithm, so that the deep learning algorithm can better extract the characteristics of time domain and frequency domain of the sEMG electromyographic signals and send the characteristics into the deep learning algorithm;
the wearable limb movement intelligent sensing system based on the electromyography is characterized in that in the sEMG electromyography signals, the electromyography data of each channel can be represented as waveform data of one measured muscle channel. Therefore, in the invention, when the deep learning model is used for semantic segmentation, the conventional spatial convolution pooling of the regular shape NxN is not adopted, but the long and narrow convolution and the strip pooling are adopted. The square sized convolutional and pooling layers limit the correlation and flexibility of upper and lower data information per channel to capture sEMG electromyographic signals in real-time hand movements.
In order to more effectively capture the correlation and flexibility of the sEMG electromyography signals on the upper and lower data information of each channel, the receiving range of a deep learning model is expanded by utilizing long and narrow convolution and stripe pooling, and the correlation characteristics of the upper and lower data of the sEMG electromyography signals are collected. In the convolutional neural network for extracting the first time-domain frequency-domain feature, a long and narrow convolution kernel with the size of 20 × 4 is used to extract the time-domain frequency-domain feature of the sEMG, according to the frequency feature of the sEMG electromyographic signal, the electromyographic frequency is mainly distributed in the range of 0-500Hz, and after digital filtering, the frequency of the sEMG electromyographic signal is completely in the frequency range of 0-500 Hz. The input sEMG electromyographic signals are convoluted by horizontal and vertical long and narrow convolution kernels to calculate H multiplied by 1 and 1 multiplied by W, the H multiplied by 1 and 1 multiplied by W are changed after long and narrow pooling is carried out, the maximum value of the element values in the pooling kernels is calculated, and the maximum value is used as a pooling output value;
in the present invention, in the above mentioned extraction of time domain features in the sEMG electromyogram signal, frequency domain features are extracted to conform to the data waveform features of the sEMG electromyogram signal, and in addition, in the second convolution layer, time domain features of the sEMG electromyogram signal are extracted by long and narrow convolution of 3 × 1 size. Therefore, the time domain and frequency domain characteristics of the sEMG electromyographic signals with different proportions in the first time domain and frequency domain characteristics are extracted, and different prediction results can be obtained after the time domain and frequency domain data with different characteristics are fully connected. Repeating the steps for 4 times in sequence, extracting time domain and frequency domain characteristics of the four sEMG electromyographic signals, and sending the characteristics into a full connection layer, wherein experiments show that the sEMG electromyographic signals have a high classification effect;
the interactive rehabilitation system based on electromyographic intelligent wearing is characterized in that the used calculation formula of the long and narrow convolution is as follows:
the interactive rehabilitation system based on electromyographic intelligent wearing is characterized in that in a deep learning model, 20 x 4 long and narrow convolution is applied firstly to extract time domain and frequency domain characteristics of sEMG electromyographic signals, then the time domain and frequency domain characteristics are subjected to maximum value banding pooling of 20 x 1, then the 3 x 1 long and narrow convolution is applied to extract time domain characteristics of sENGl, and the time domain characteristics are subjected to maximum value banding pooling of 7 x 1. In the process of extracting the time-frequency domain characteristics of the first sEMG electromyographic signal, the frequency distribution of each action is subjected to characteristic fusion and dimension reduction for multiple times, and after the stripe pooling layer is applied, the Maxpooling layer has local invariance and can extract obvious characteristics and reduce parameters of the model, so that overfitting of the model is reduced. The Maxpooling only extracts the significant features of the sEMG electromyographic signals and discards insignificant information, the extracted time-frequency domain features can effectively represent the multivariate time sequence of the sEMG electromyographic signals, and the generation of overfitting can be relieved to a certain extent due to the fact that the parameters of the model are reduced.
8. An application of the interactive rehabilitation system based on electromyographic intelligence wearing according to claim 1, a remote unmanned aerial vehicle control module, comprising:
(1) the result output by the physiological status analysis module (2) of claim 7 is sent to control the bionic manipulator control module (4) through a Bluetooth wireless transmission circuit (17).
9. An application of the interactive rehabilitation system worn based on electromyographic intelligence according to claim 1, comprising:
(1) in order to ensure that the electrode paste is completely contacted with the skin; before the electrode paste is pasted, firstly, wiping the skin to be detected of the dominant hand of the wearer by alcohol cotton, keeping the wearer sitting still, annotating a screen on the eyes, and finishing the action required by the active training interface of the hand myoelectricity; after a system power supply circuit (16) is ensured to be normal, the system is started, sEMG electromyographic signals are collected through wearable electromyographic collection equipment and are transmitted to a physiological state analysis module, and electromyographic collection work is completed;
(2) the physiological state analysis module automatically carries out digital filtering on the collected sEMG electromyographic signals;
(3) the classification and identification system performs classification and identification according to the digitized filtered sEMG electromyographic signals based on a pre-trained deep convolutional neural network model;
(4) and sending the classification result to a signal control bionic manipulator control module (4) through a Bluetooth wireless transmission circuit (17).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110631860.5A CN113871028A (en) | 2021-06-07 | 2021-06-07 | Interactive rehabilitation system based on myoelectric intelligent wearing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110631860.5A CN113871028A (en) | 2021-06-07 | 2021-06-07 | Interactive rehabilitation system based on myoelectric intelligent wearing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113871028A true CN113871028A (en) | 2021-12-31 |
Family
ID=78989974
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110631860.5A Pending CN113871028A (en) | 2021-06-07 | 2021-06-07 | Interactive rehabilitation system based on myoelectric intelligent wearing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113871028A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114546111A (en) * | 2022-01-30 | 2022-05-27 | 天津大学 | Myoelectricity-based intelligent trolley hand wearing control system and application |
| CN117373614A (en) * | 2023-12-07 | 2024-01-09 | 深圳市微克科技有限公司 | Finger health training method, system and storage medium based on wearing product |
-
2021
- 2021-06-07 CN CN202110631860.5A patent/CN113871028A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114546111A (en) * | 2022-01-30 | 2022-05-27 | 天津大学 | Myoelectricity-based intelligent trolley hand wearing control system and application |
| CN114546111B (en) * | 2022-01-30 | 2023-09-08 | 天津大学 | Myoelectricity-based intelligent trolley hand wearing control system and application |
| CN117373614A (en) * | 2023-12-07 | 2024-01-09 | 深圳市微克科技有限公司 | Finger health training method, system and storage medium based on wearing product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111631907B (en) | Cerebral apoplexy patient hand rehabilitation system based on brain-computer interaction hybrid intelligence | |
| Ferreira et al. | Human-machine interfaces based on EMG and EEG applied to robotic systems | |
| Milosevic et al. | Design challenges for wearable EMG applications | |
| CN100594858C (en) | Electric artificial hand combined controlled by brain electricity and muscle electricity and control method | |
| Brunelli et al. | Low-cost wearable multichannel surface EMG acquisition for prosthetic hand control | |
| CN107788976A (en) | Sleep monitor system based on Amplitude integrated electroencephalogram | |
| Jiang et al. | IoT-based remote facial expression monitoring system with sEMG signal | |
| CN112022619B (en) | Multi-mode information fusion sensing system of upper limb rehabilitation robot | |
| CN111317600B (en) | Artificial limb control method, device, system, equipment and storage medium | |
| CN113871028A (en) | Interactive rehabilitation system based on myoelectric intelligent wearing | |
| CN114647314A (en) | Wearable limb movement intelligent sensing system based on myoelectricity | |
| CN111696645A (en) | Hand exoskeleton rehabilitation training device and method based on surface electromyographic signals | |
| CN111513735A (en) | Major depressive disorder identification system based on brain-computer interface and deep learning and application | |
| WO2019132692A1 (en) | Method and system for controlling electronic devices with the aid of an electromyographic reading device | |
| CN113855053A (en) | Wearable muscle threshold monitoring system based on myoelectricity | |
| CN104267807A (en) | Hand action mechanomyography based man-machine interaction method and interaction system | |
| CN107126303A (en) | A kind of upper and lower extremities exercising support method based on mobile phone A PP | |
| CN113143676A (en) | Control method of external limb finger based on brain-muscle-electricity cooperation | |
| Pinzón-Arenas et al. | Percentage estimation of muscular activity of the forearm by means of EMG signals based on the gesture recognized using CNN | |
| Chang et al. | A hierarchical hand motions recognition method based on IMU and sEMG sensors | |
| Boonarchatong et al. | Green EEG energy control robot for supporting bedfast patients | |
| CN114504333B (en) | Wearable vestibule monitoring system based on myoelectricity and application | |
| CN201227336Y (en) | Electric artificial hand controlled by brain electricity and muscle electricity | |
| CN110751060B (en) | Portable motion mode real-time identification system based on multi-source signals | |
| Salvekar et al. | Mind controlled robotic arm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |