CN111950460B - Muscle strength self-adaptive stroke patient hand rehabilitation training action recognition method - Google Patents
Muscle strength self-adaptive stroke patient hand rehabilitation training action recognition method Download PDFInfo
- Publication number
- CN111950460B CN111950460B CN202010811227.XA CN202010811227A CN111950460B CN 111950460 B CN111950460 B CN 111950460B CN 202010811227 A CN202010811227 A CN 202010811227A CN 111950460 B CN111950460 B CN 111950460B
- Authority
- CN
- China
- Prior art keywords
- data
- convolution
- pooling
- size
- rehabilitation training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/30—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/02—Preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Signal Processing (AREA)
- Physical Education & Sports Medicine (AREA)
- Epidemiology (AREA)
- Medical Informatics (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention discloses a stroke patient hand rehabilitation training action recognition method capable of adapting to muscle force, which is used for simultaneously acquiring electromyographic signals, acceleration and angular velocity data, simultaneously obtaining physiological and kinematic objective measurement data, and using sample set training models at different muscle force levels, so that the deep learning model can adapt to the change of the muscle force of a subject when the subject finishes action, and the problem of low recognition robustness caused by larger difference of the electromyographic signals under different muscle force conditions is solved. According to the method, different data characteristics are automatically and adaptively extracted through the multi-path multi-size convolutional neural network, the neural network is used for classification and identification after various characteristics are fused, information redundancy or loss and characteristic weakening caused by artificial characteristic engineering are avoided, and the identification precision of the hand rehabilitation training action of the stroke patient at different muscle strength levels is effectively improved.
Description
Technical Field
The invention relates to the field of rehabilitation action recognition, in particular to a muscle strength adaptive stroke patient hand rehabilitation training action recognition method.
Background
The rehabilitation training utilizes the plasticity of the central system to help the stroke patient to recover the motion function to a certain extent, has great significance for the recognition of the rehabilitation training action of the patient, and the recognition result can be clinically used as a control signal of auxiliary training equipment to control the motion of artificial limbs and assist the limb disorder patient to complete the motion function similar to real limbs; or the training action recognition result is used as the basis for the evaluation of the motion function; and intelligent rehabilitation training can be realized in remote interactive rehabilitation or the training condition can be monitored remotely by assisting doctors.
Most of the existing rehabilitation training action recognition methods based on surface electromyography acquire signals through multi-channel surface electromyography, manually extract data characteristics, and then apply machine learning algorithm to carry out classification recognition. Such methods have the following drawbacks:
1) the hand rehabilitation training action is fine, a large amount of muscles are needed to be cooperated to complete, muscle activity data cannot be comprehensively acquired only by using multi-channel surface myoelectricity, the cooperation of the muscles is ignored, and the precision of classification and identification is influenced;
2) the data is subjected to manual feature extraction, so that insufficient or redundant feature extraction can be caused by subjective experience, and the performance of a subsequent algorithm is influenced;
3) the currently used machine learning classification algorithm (such as a support vector machine, an extreme learning machine, a Gaussian mixture model, and the like) is poor in recognition classification performance for classification tasks with small data differences such as hand rehabilitation training actions.
Disclosure of Invention
Aiming at the defects in the prior art, the recognition method for the hand rehabilitation training action of the stroke patient with the self-adaptive muscle strength provided by the invention solves the problem of poor recognition accuracy of the existing method.
In order to achieve the purpose of the invention, the invention adopts the technical scheme that:
the method for recognizing the hand rehabilitation training action of the stroke patient with the self-adaptive muscle strength comprises the following steps:
s1, placing a circle of myoelectric electrodes around the forearm at the forearm muscle group position of the patient, placing an inertial sensor at the wrist of the back of the hand of the patient, and acquiring myoelectric data and inertial sensor data of the patient during rehabilitation training; wherein the inertial sensor data comprises acceleration data and angular velocity data;
s2, filtering and rectifying the electromyographic data, and then normalizing to obtain an electromyographic data normalization result; performing Kalman filtering on the acceleration data and the angular velocity data to obtain filtered acceleration data and filtered angular velocity data;
s3, respectively acquiring the characteristics of the electromyographic data normalization result, the filtered acceleration data and the filtered angular velocity data by adopting a three-way convolutional neural network;
s4, splicing the characteristics of the electromyographic data normalization result, the characteristics of the filtered acceleration data and the characteristics of the filtered angular velocity data to obtain an integral characteristic vector;
s5, acquiring classification numerical values of the integral feature vectors by adopting two fully-connected layers;
and S6, converting the classification numerical values into probability distribution to obtain probability values of the rehabilitation training actions belonging to all the classes.
Further, the specific method of step S1 is:
a cycle of myoelectric electrodes are arranged around the forearm at the forearm muscle group position of a patient, the channels are 8, and the sampling frequency is 1 KHz; placing an inertial sensor comprising a triaxial acceleration sensing unit and a triaxial gyroscope sensing unit at the wrist of the back of a hand of a patient, wherein 6 channels are formed in total, and the sampling frequency is 20 Hz; acquiring myoelectric data, acceleration data and angular velocity data of a patient during rehabilitation training; the Z axis of the inertial sensor is perpendicular to the plane of the back of the hand, the Y axis is along the direction of the fingers, and the X axis points to the left direction of the patient.
Further, in step S2, the electromyographic data is normalized after being filtered and rectified, and a specific method for obtaining the electromyographic data normalization result includes:
adopting a 50Hz notch filter to filter the electromyographic data, adopting a 20Hz-450Hz band-pass filter to filter the filtered data again, and carrying out full-wave rectification on the data after the filtering again according to a formula:
carrying out dispersion normalization on the data X after full-wave rectification to obtain an electromyographic data normalization result X * (ii) a Wherein min (X) represents the minimum value in the full-wave rectified data X; max (X) represents the maximum value in the full-wave rectified data X.
Further, in the step S3, the three convolutional neural networks have the same structure, and each convolutional neural network includes three convolutional pooling layers and two multi-size convolutional blocks which are connected in sequence; wherein the first convolution pooling layer comprises 32 convolution kernels of size 7 × 1 and pooling kernels of size 5 × 1; the second convolution pooling layer includes 64 convolution kernels of size 5 × 1 and convolution kernels of size 3 × 1Pooling kernels; the third convolution pooling layer comprises 128 convolution kernels of size 3 × 1 and pooling kernels of size 3 × 1; the convolution operation formula of the kth convolution pooling layer is as follows:
outputting the convolution result of the kth convolution pooling layer;
outputting the convolution result of the k-1 layers of convolution pooling layers;
a convolution kernel that is a kth convolution pooling layer;
is an offset; n is a radical of hydrogen k-1 Inputting a total channel for the convolution layer of the kth convolution pooling layer; i is a channel index of the input data; j is the number of convolution kernels of the kth convolution pooling layer; ReLU (-) is a ReLU activation function;
the first layer of multi-size convolution block comprises four paths, the input of each path is the same pooling kernel with the size of 3 multiplied by 1, the first path also comprises 16 convolution kernels with the size of 1 multiplied by 1, 16 convolution kernels with the size of 3 multiplied by 1 and 32 convolution kernels with the size of 3 multiplied by 1 after the pooling kernel, and feature vectors of 32 channels are output; the second path also comprises 96 convolution kernels of 1 × 1 and 128 convolution kernels of 3 × 1 after the pooling kernel, and outputs a feature vector of 128 channels; the third path also comprises 1 3 × 1 pooling kernel and 32 1 × 1 convolution kernels after the pooling kernel, and outputs a feature vector of 32 channels; the fourth path also comprises 64 convolution kernels of 1 multiplied by 1 after the pooling kernel, and feature vectors of 64 channels are output; outputting feature vectors of 256 channels by the first layer multi-size volume block;
the second layer of multi-size convolution block comprises four paths, the input of each path is the same pooling kernel with the size of 3 multiplied by 1, the first path also comprises 32 convolution kernels with the size of 1 multiplied by 1, 32 convolution kernels with the size of 3 multiplied by 1 and 96 convolution kernels with the size of 3 multiplied by 1 after the pooling kernel, and the feature vectors of 96 channels are output; the second path also comprises 128 convolution kernels of 1 × 1 and 192 convolution kernels of 3 × 1 after the pooling kernel, and outputs feature vectors of 192 channels; the third path also comprises 1 3 × 1 pooling kernel and 64 1 × 1 convolution kernels after the pooling kernel, and 64 channels of feature vectors are output; the fourth path also comprises 128 convolution kernels of 1 × 1 after the pooling kernel, and outputs a feature vector of 128 channels; the second layer of multi-size convolution block outputs feature vectors of 480 channels, namely each convolution neural network outputs feature vectors of 480 channels.
Further, the specific method of step S4 is:
and sequentially splicing the characteristics of the electromyographic data normalization result, the characteristics of the filtered acceleration data and the characteristics of the filtered angular velocity data to obtain an integral characteristic vector.
Further, in step S5, the first fully-connected layer includes 1024 neurons, and the number of neurons in the second fully-connected layer is equal to the number of rehabilitation training action categories.
Further, the specific method of step S6 is:
according to the formula:
classifying the numerical value q by adopting softmax layer n Conversion into a probability value S belonging to the nth action category n Further obtaining probability values of the rehabilitation training actions belonging to all categories; wherein e is a constant, and N is the total number of elements in the classification numerical value, namely the number of rehabilitation training action categories.
The invention has the beneficial effects that: the invention simultaneously and objectively measures the muscle activity electric signal and the kinematic data of a stroke patient in the rehabilitation training process, constructs a multi-path convolution neural network, and adaptively and automatically extracts the characteristics of different sensing data; the depth and the width of the network are increased through the multi-size convolution block, the data fitting capacity of the model is improved, and the features are fully extracted; training the model by using the sample sets under different muscle strength levels, so that the recognition effect of the model under the condition of muscle strength change is more stable; the hand rehabilitation training action recognition of the stroke patient with the end-to-end self-adaptive muscle strength is realized by fusing the characteristics of different sensing data, and the recognition precision is improved.
Drawings
FIG. 1 is a schematic flow diagram of the present invention;
FIG. 2 is a block diagram of a network architecture of the present invention;
fig. 3 is a block diagram of a network structure of a multi-sized volume block.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined in the appended claims, and all matters produced by the invention using the inventive concept are protected.
As shown in fig. 1, the method for recognizing the hand rehabilitation training action of the stroke patient with adaptive muscle strength is characterized by comprising the following steps:
s1, placing a circle of myoelectric electrodes around the forearm at the forearm muscle group position of the patient, placing an inertial sensor at the wrist of the back of the hand of the patient, and acquiring myoelectric data and inertial sensor data of the patient during rehabilitation training; wherein the inertial sensor data comprises acceleration data and angular velocity data;
s2, filtering and rectifying the electromyographic data, and then normalizing to obtain an electromyographic data normalization result; performing Kalman filtering on the acceleration data and the angular velocity data to obtain filtered acceleration data and filtered angular velocity data;
s3, respectively acquiring the characteristics of the electromyographic data normalization result, the filtered acceleration data and the filtered angular velocity data by adopting a three-way convolutional neural network;
s4, splicing the characteristics of the electromyographic data normalization result, the characteristics of the filtered acceleration data and the characteristics of the filtered angular velocity data to obtain an integral characteristic vector;
s5, acquiring classification numerical values of the integral feature vectors by adopting two fully-connected layers;
and S6, converting the classification numerical values into probability distribution to obtain probability values of the rehabilitation training actions belonging to all the classes.
The specific method of step S1 is: placing a circle of myoelectric electrodes around the forearm at the forearm muscle group position (3-4 cm away from the elbow joint) of a patient, wherein the total number of channels is 8, and the sampling frequency is 1 KHz; placing an inertial sensor comprising a triaxial acceleration sensing unit and a triaxial gyroscope sensing unit at the wrist of the back of a hand of a patient, wherein 6 channels are formed in total, and the sampling frequency is 20 Hz; acquiring myoelectric data, acceleration data and angular velocity data of a patient during rehabilitation training; the Z axis of the inertial sensor is perpendicular to the plane of the back of the hand, the Y axis is along the direction of the fingers, and the X axis points to the left direction of the patient.
In step S2, the electromyographic data is normalized after being filtered and rectified, and the specific method for obtaining the electromyographic data normalization result is as follows: adopting a 50Hz notch filter to filter the electromyographic data, adopting a 20Hz-450Hz band-pass filter to filter the filtered data again, and carrying out full-wave rectification on the data after the filtering again according to a formula:
carrying out dispersion normalization on the data X after full-wave rectification to obtain an electromyographic data normalization result X * (ii) a Wherein min (X) represents the minimum value in the full-wave rectified data X; max (X) represents the maximum value in the full-wave rectified data X.
As shown in fig. 2 and 3, the three convolutional neural networks in step S3 have the same structure, and each convolutional neural network includes three convolutional pooling layers and two multi-size convolutional blocks connected in sequence; wherein the first convolution pooling layer comprises 32 convolution kernels of size 7 × 1 and pooling kernels of size 5 × 1; the second convolution pooling layer includes 64 convolution kernels of size 5 × 1 and pooling kernels of size 3 × 1; third layer of convolution poolingA layer includes 128 convolution kernels of size 3 x 1 and pooling kernels of size 3 x 1; the convolution operation formula of the kth convolution pooling layer is as follows:
outputting the convolution result of the kth convolution pooling layer;
outputting the convolution result of the k-1 layers of convolution pooling layers;
a convolution kernel that is a kth convolution pooling layer;
is an offset; n is a radical of k-1 Inputting a total channel for the convolution layer of the kth convolution pooling layer; i is a channel index of the input data; j is the number of convolution kernels of the kth convolution pooling layer; ReLU (·) is a ReLU activation function;
the first layer of multi-size convolution block comprises four paths, the input of each path is the same pooling kernel with the size of 3 multiplied by 1, the first path also comprises 16 convolution kernels with the size of 1 multiplied by 1, 16 convolution kernels with the size of 3 multiplied by 1 and 32 convolution kernels with the size of 3 multiplied by 1 after the pooling kernel, and feature vectors of 32 channels are output; the second path also comprises 96 convolution kernels of 1 × 1 and 128 convolution kernels of 3 × 1 after the pooling kernel, and outputs a feature vector of 128 channels; the third path also comprises 1 3 × 1 pooling kernel and 32 1 × 1 convolution kernels after the pooling kernel, and outputs a feature vector of 32 channels; the fourth path also comprises 64 convolution kernels of 1 multiplied by 1 after the pooling kernel, and feature vectors of 64 channels are output; outputting feature vectors of 256 channels by the first layer multi-size volume block;
the second layer of multi-size convolution block comprises four paths, the input of each path is the same pooling kernel with the size of 3 multiplied by 1, the first path also comprises 32 convolution kernels with the size of 1 multiplied by 1, 32 convolution kernels with the size of 3 multiplied by 1 and 96 convolution kernels with the size of 3 multiplied by 1 after the pooling kernel, and the feature vectors of 96 channels are output; the second path also comprises 128 convolution kernels of 1 × 1 and 192 convolution kernels of 3 × 1 after the pooling kernel, and outputs feature vectors of 192 channels; the third path also comprises 1 3 × 1 pooling kernel and 64 1 × 1 convolution kernels after the pooling kernel, and 64 channels of feature vectors are output; the fourth path also comprises 128 convolution kernels of 1 × 1 after the pooling kernel, and outputs a feature vector of 128 channels; the second layer multi-size convolution block outputs feature vectors of 480 channels, namely each path of convolution neural network outputs feature vectors of 480 channels.
According to the scheme, the multi-size volume blocks are introduced, the depth and the width of the network are increased, the adaptability of the network to the scale is also increased, the characteristics of data in different scales can be learned, and the network performance is improved. The operations of convolution, activation function and pooling in the multi-size volume block are the same as the aforementioned equations. In order to ensure that a plurality of paths of each multi-size convolution block can carry out channel combination on an output layer, the dimension of output data of each path is required to be the same, so that a parameter padding is set to be 1 when all filters (including convolution and pooling) with the size of 3 x 1 of each path are operated, namely, a boundary with a numerical value of 0 is added before and after sequence data when convolution or pooling is carried out, and thus the number of the paths of input and output data is changed without changing the dimension.
The specific method of step S4 is: and sequentially splicing the characteristics of the electromyographic data normalization result, the characteristics of the filtered acceleration data and the characteristics of the filtered angular velocity data to obtain an integral characteristic vector. In step S5, the first fully-connected layer includes 1024 neurons, and the number of neurons in the second fully-connected layer is equal to the number of rehabilitation training action categories.
The specific method of step S6 is: according to the formula:
classifying the value q by adopting softmax layer n Conversion into a probability value S belonging to the nth action category n Further obtaining probability values of the rehabilitation training actions belonging to all categories; wherein e isAnd N is the total number of elements in the classification numerical value, namely the number of rehabilitation training action categories.
In one embodiment of the present invention, there are a total of 10 types of hand rehabilitation exercises designed, as shown in table 1. The patient relaxes and sits on the armchair, the double-arm support is placed at a fixed position on a table, according to video or voice instructions, the actions are respectively completed at three muscle strength (muscle contraction) levels of the lowest, the medium and the highest of the individual, the time from rest to contraction is caused, the posture is kept for 5 seconds, the same action at the same muscle strength level is repeated for 5 times in each experiment, and the interval between the actions is 10 seconds.
Table 1: rehabilitation training action design
Acquiring 8-channel myoelectricity E ═ E { E } in the data acquisition stage 1 ，e 2 ，...，e 8 Triple channel acceleration A ═ a 1 ，a 2 ，a 3 And three-channel angular velocity G ═ G 1 ，g 2 ，g 3 The three types of data are manually selected and divided into the same 5-second time period, and the data lengths are respectively T E 、T A And T G . The collected electromyographic data is subjected to filtering pretreatment, a 50Hz notch filter is used for removing power line interference noise, and 20Hz-450Hz band-pass filtering is used for removing motion artifacts (< 20Hz) and high-frequency noise (f: (20 Hz))>450Hz) and full-wave rectified. And performing Kalman filtering on the acceleration and angular velocity data to remove noise. ReLU function as activation function for convolutional layer: relu (x) max (0, x); the pooling layer of the scheme uses maximum pooling, extracts the significant features of the data and reduces the dimension of the data.
In the implementation process of the invention, the cross entropy can be adopted to calculate the classification error, namely the measurement distance between the sample prediction and the real action class probability distribution, and the cost function is：
Wherein Batch is the number of Batch samples selected in the Batch stochastic gradient descent; o is g Predicting probability distribution, y, for the calculation of the g-th sample g Is the true probability distribution of the g-th sample. And (3) iterating the parameters in the network by using an error back propagation algorithm with batch gradient reduction based on the obtained error value of the output and the actual output, and finishing parameter training and storing after at least 5000 iterations.
In conclusion, the method collects the electromyographic signals, the acceleration data and the angular velocity data at the same time, obtains physiological and kinematic objective measurement data at the same time, and trains the model by using the sample set under different muscle strength levels, so that the deep learning model can be self-adapted to the change of the muscle strength of the testee when finishing the action, and the problem of low recognition robustness caused by large difference of the electromyographic signals under different muscle strength conditions is solved. According to the method, different data characteristics are automatically and adaptively extracted through the multi-path multi-size convolutional neural network, the neural network is used for classification and identification after various characteristics are fused, information redundancy or loss and characteristic weakening caused by artificial characteristic engineering are avoided, and the identification precision of the hand rehabilitation training action of the stroke patient at different muscle strength levels is effectively improved.
Claims (6)
1. A hand rehabilitation training action recognition method of a stroke patient with self-adaptive muscle strength is characterized by comprising the following steps:
s1, placing a circle of myoelectric electrodes around the forearm at the forearm muscle group position of the patient, placing an inertial sensor at the wrist of the back of the hand of the patient, and acquiring myoelectric data and inertial sensor data of the patient during rehabilitation training; wherein the inertial sensor data comprises acceleration data and angular velocity data;
s2, filtering and rectifying the electromyographic data, and then normalizing to obtain an electromyographic data normalization result; performing Kalman filtering on the acceleration data and the angular velocity data to obtain filtered acceleration data and filtered angular velocity data;
s3, respectively acquiring the characteristics of the electromyographic data normalization result, the filtered acceleration data and the filtered angular velocity data by adopting a three-way convolutional neural network;
s4, splicing the characteristics of the electromyographic data normalization result, the characteristics of the filtered acceleration data and the characteristics of the filtered angular velocity data to obtain an integral characteristic vector;
s5, acquiring classification numerical values of the integral feature vectors by adopting two fully-connected layers;
s6, converting the classification values into probability distribution to obtain probability values of rehabilitation training actions belonging to various categories;
in the step S3, the three convolutional neural networks have the same structure, and each convolutional neural network comprises three convolutional pooling layers and two layers of multi-size convolutional blocks which are sequentially connected; wherein the first convolution pooling layer comprises 32 convolution kernels of size 7 × 1 and pooling kernels of size 5 × 1; the second convolution pooling layer includes 64 convolution kernels of size 5 × 1 and pooling kernels of size 3 × 1; the third convolution pooling layer comprises 128 convolution kernels of size 3 × 1 and pooling kernels of size 3 × 1; the convolution operation formula of the kth convolution pooling layer is as follows:
outputting the convolution result of the kth convolution pooling layer;
outputting the convolution result of the k-1 layers of convolution pooling layers;
a convolution kernel that is a kth convolution pooling layer;
is an offset; n is a radical of k-1 Convolution layer being convolution pooling layer of k-th layerInputting a main channel; i is a channel index of the input data; j is the convolution kernel number of the k-th convolution pooling layer; ReLU (·) is a ReLU activation function;
the first layer of multi-size convolution block comprises four paths, the input of each path is the same pooling kernel with the size of 3 multiplied by 1, the first path also comprises 16 convolution kernels with the size of 1 multiplied by 1, 16 convolution kernels with the size of 3 multiplied by 1 and 32 convolution kernels with the size of 3 multiplied by 1 after the pooling kernel, and feature vectors of 32 channels are output; the second path also comprises 96 convolution kernels of 1 × 1 and 128 convolution kernels of 3 × 1 after the pooling kernel, and outputs a feature vector of 128 channels; the third path also comprises 1 3 × 1 pooling kernel and 32 1 × 1 convolution kernels after the pooling kernel, and outputs a feature vector of 32 channels; the fourth path also comprises 64 convolution kernels of 1 multiplied by 1 after the pooling kernel, and feature vectors of 64 channels are output; outputting feature vectors of 256 channels by the first layer multi-size volume block;
the second layer of multi-size convolution block comprises four paths, the input of each path is the same pooling kernel with the size of 3 multiplied by 1, the first path also comprises 32 convolution kernels with the size of 1 multiplied by 1, 32 convolution kernels with the size of 3 multiplied by 1 and 96 convolution kernels with the size of 3 multiplied by 1 after the pooling kernel, and the feature vectors of 96 channels are output; the second path also comprises 128 convolution kernels of 1 × 1 and 192 convolution kernels of 3 × 1 after the pooling kernel, and outputs feature vectors of 192 channels; the third path also comprises 1 3 × 1 pooling kernel and 64 1 × 1 convolution kernels after the pooling kernel, and 64 channels of feature vectors are output; the fourth path also comprises 128 convolution kernels of 1 × 1 after the pooling kernel, and outputs a feature vector of 128 channels; the second layer multi-size convolution block outputs feature vectors of 480 channels, namely each path of convolution neural network outputs feature vectors of 480 channels.
2. The method for recognizing hand rehabilitation training actions of stroke patients with adaptive muscle strength as claimed in claim 1, wherein the specific method in step S1 is as follows:
a cycle of myoelectric electrodes are arranged around the forearm at the forearm muscle group position of a patient, the channels are 8, and the sampling frequency is 1 KHz; placing an inertial sensor comprising a triaxial acceleration sensing unit and a triaxial gyroscope sensing unit at the wrist of the back of a hand of a patient, wherein 6 channels are formed in total, and the sampling frequency is 20 Hz; acquiring myoelectric data, acceleration data and angular velocity data of a patient during rehabilitation training; the Z axis of the inertial sensor is perpendicular to the plane of the back of the hand, the Y axis is along the direction of the fingers, and the X axis points to the left direction of the patient.
3. The method for recognizing the hand rehabilitation training action of the patient with stroke with adaptive muscle strength according to claim 1, wherein the myoelectric data is normalized after being filtered and rectified in step S2, and the specific method for obtaining the normalization result of the myoelectric data is as follows:
adopting a 50Hz notch filter to filter the electromyographic data, adopting a 20Hz-450Hz band-pass filter to filter the filtered data again, and carrying out full-wave rectification on the data after the filtering again according to a formula:
carrying out dispersion normalization on the data X after full-wave rectification to obtain an electromyographic data normalization result X * (ii) a Wherein min (X) represents the minimum value in the full-wave rectified data X; max (X) represents the maximum value in the full-wave rectified data X.
4. The method for recognizing hand rehabilitation training actions of stroke patients with adaptive muscle strength as claimed in claim 1, wherein the specific method in step S4 is as follows:
and sequentially splicing the characteristics of the electromyographic data normalization result, the characteristics of the filtered acceleration data and the characteristics of the filtered angular velocity data to obtain an integral characteristic vector.
5. The adaptive muscle strength stroke patient hand rehabilitation training action recognition method as claimed in claim 1, wherein in step S5, the first fully-connected layer comprises 1024 neurons, and the number of the neurons in the second fully-connected layer is equal to the number of rehabilitation training action categories.
6. The method for recognizing hand rehabilitation training actions of stroke patients with adaptive muscle strength as claimed in claim 1, wherein the specific method in step S6 is as follows:
according to the formula:
classifying the value q by adopting softmax layer n Conversion into a probability value S belonging to the nth action category n Further obtaining probability values of the rehabilitation training actions belonging to all categories; wherein e is a constant, and N is the total number of elements in the classification numerical value, namely the number of rehabilitation training action categories.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010811227.XA CN111950460B (en) | 2020-08-13 | 2020-08-13 | Muscle strength self-adaptive stroke patient hand rehabilitation training action recognition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010811227.XA CN111950460B (en) | 2020-08-13 | 2020-08-13 | Muscle strength self-adaptive stroke patient hand rehabilitation training action recognition method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111950460A CN111950460A (en) | 2020-11-17 |
| CN111950460B true CN111950460B (en) | 2022-09-20 |
Family
ID=73331727
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010811227.XA Active CN111950460B (en) | 2020-08-13 | 2020-08-13 | Muscle strength self-adaptive stroke patient hand rehabilitation training action recognition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111950460B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113705511A (en) * | 2021-09-02 | 2021-11-26 | 广州市奥威亚电子科技有限公司 | Gesture recognition method and device |
| CN113712541B (en) * | 2021-09-03 | 2022-08-02 | 上海诺诚电气股份有限公司 | Rehabilitation method and system based on multi-sensor action recognition |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106293057A (en) * | 2016-07-20 | 2017-01-04 | 西安中科比奇创新科技有限责任公司 | Gesture identification method based on BP neutral net |
| CN106377253A (en) * | 2016-09-14 | 2017-02-08 | 燕山大学 | Electromyographic signal and inertia information synchronous collection device |
| CN107220657B (en) * | 2017-05-10 | 2018-05-18 | 中国地质大学（武汉） | A kind of method of high-resolution remote sensing image scene classification towards small data set |
| CN108932452A (en) * | 2017-05-22 | 2018-12-04 | 中国科学院半导体研究所 | Arrhythmia cordis beat classification method based on multiple dimensioned convolutional neural networks |
| CN109214250A (en) * | 2017-07-05 | 2019-01-15 | 中南大学 | A kind of static gesture identification method based on multiple dimensioned convolutional neural networks |
| CN109784412A (en) * | 2019-01-23 | 2019-05-21 | 复旦大学 | The multiple sensor signals fusion method based on deep learning for gait classification |
| CN109875565A (en) * | 2019-01-25 | 2019-06-14 | 电子科技大学 | A kind of cerebral apoplexy upper extremity exercise function method for automatically evaluating based on deep learning |
| CN111046731B (en) * | 2019-11-11 | 2023-07-25 | 中国科学院计算技术研究所 | Transfer learning method and recognition method for gesture recognition based on surface electromyographic signals |
| CN111184512B (en) * | 2019-12-30 | 2021-06-01 | 电子科技大学 | Method for recognizing rehabilitation training actions of upper limbs and hands of stroke patient |
| CN111242100B (en) * | 2020-03-05 | 2023-02-07 | 合肥工业大学 | Action identification method based on GST and VL-MOBPNN |
-
2020
- 2020-08-13 CN CN202010811227.XA patent/CN111950460B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111950460A (en) | 2020-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110537922B (en) | Human body walking process lower limb movement identification method and system based on deep learning | |
| US6660042B1 (en) | Methods of biomimetic finger control by filtering of distributed forelimib pressures | |
| CN111950460B (en) | Muscle strength self-adaptive stroke patient hand rehabilitation training action recognition method | |
| CN109009586B (en) | Myoelectric continuous decoding method for man-machine natural driving angle of artificial wrist joint | |
| CN108983973B (en) | Control method of humanoid smart myoelectric artificial hand based on gesture recognition | |
| CN109620651B (en) | Intelligent auxiliary rehabilitation equipment based on synchronous brain and muscle electricity | |
| CN102622605A (en) | Surface electromyogram signal feature extraction and action pattern recognition method | |
| CN112043473B (en) | Parallel nested and autonomous preferred classifier for brain-myoelectricity fusion perception of intelligent artificial limb | |
| CN112603758A (en) | Gesture recognition method based on sEMG and IMU information fusion | |
| Bu et al. | EMG-based motion discrimination using a novel recurrent neural network | |
| CN109446972B (en) | Gait recognition model establishing method, recognition method and device based on electromyographic signals | |
| CN113534960B (en) | Upper arm artificial limb control method and system based on IMU and surface electromyographic signals | |
| CN112949676B (en) | Self-adaptive motion mode identification method of flexible lower limb assistance exoskeleton robot | |
| CN112541415B (en) | Brain muscle function network motion fatigue detection method based on symbol transfer entropy and graph theory | |
| Chu et al. | Control of multifunction myoelectric hand using a real-time EMG pattern recognition | |
| Kaysa et al. | Design of Brain-computer interface platform for semi real-time commanding electrical wheelchair simulator movement | |
| CN111643321A (en) | Exoskeleton joint angle prediction method and system based on sEMG signals | |
| Pancholi et al. | Intelligent upper-limb prosthetic control (iULP) with novel feature extraction method for pattern recognition using EMG | |
| CN111938660B (en) | Stroke patient hand rehabilitation training action recognition method based on array myoelectricity | |
| CN113171214A (en) | Multi-path feedback myoelectricity control prosthetic hand based on self-adaptive enhanced classifier and method | |
| CN109758336A (en) | A kind of control method carrying out active rehabilitation care using patient's Ipsilateral sEMG | |
| CN102764167A (en) | Myoelectric prosthesis control source lead optimization method based on correlation coefficients | |
| Chu et al. | A real-time EMG pattern recognition based on linear-nonlinear feature projection for multifunction myoelectric hand | |
| CN105769186B (en) | Electromyographic signal collection position choosing method based on complex network | |
| Potluri et al. | Frequency domain surface EMG sensor fusion for estimating finger forces |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |