CN112232161B - Complex motion continuous estimation method based on electromyography mapping model switching - Google Patents
Complex motion continuous estimation method based on electromyography mapping model switching Download PDFInfo
- Publication number
- CN112232161B CN112232161B CN202011066204.7A CN202011066204A CN112232161B CN 112232161 B CN112232161 B CN 112232161B CN 202011066204 A CN202011066204 A CN 202011066204A CN 112232161 B CN112232161 B CN 112232161B
- Authority
- CN
- China
- Prior art keywords
- estimation
- lstm
- model
- electromyographic
- signals
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/02—Preprocessing
- G06F2218/04—Denoising
Abstract
The invention discloses a complex motion continuous estimation method based on electromyography mapping model switching, which belongs to the field of man-machine interaction. The invention effectively improves the estimation precision of each joint angle under complex tasks.
Description
Technical Field
The invention belongs to the field of human-computer interaction, and particularly relates to a complex motion continuous estimation method based on switching of an electromyography mapping model.
Background
Currently, the myoelectric interface is the only clinically feasible form of neural interface. It uses the surface muscle electromyogram signal as a signal source to estimate the movement of each joint of the human body, which has become a clinically feasible solution, but only the movement pattern recognition based on the electromyogram signal is really put into application. Many students have invested a lot of research and obtained certain achievements in motion continuous estimation and control of external equipment (such as a mechanical arm, a mechanical arm and an exoskeleton) based on electromyographic signals, but still many problems exist when the students are actually invested in clinical application, and one of the main problems is that the functions of the electromyographic-controlled external equipment cannot adapt to complex tasks in real life, for example, people can easily allocate each joint of an upper limb to complete a series of actions of extending an arm (reaching the position of a water cup by a hand), grasping (grasping the water cup by fingers), bending the arm (taking back the water cup), lifting the arm and turning over the wrist (drinking water) and the like, and the task of drinking water is completed. However, the series of actions involves precise fitting and scheduling of multiple joints in time and space. For another example, in daily life, people sometimes need to grip a soft and light object such as a paper cup, sometimes need to grip a hard and heavy object such as a pan handle, and can self-adaptively grip the paper cup with different forces according to different gripping objects.
The task is complicated due to the change of factors such as force, posture, speed and the like, particularly, the task is difficult to be continuously estimated by using the electromyographic signals, meanwhile, in the complicated task, different muscle groups are activated by a human body at different time or space to drive different joint motions, and the mapping relation between the electromyographic signals of all stages and the joint motions is different. Therefore, a single estimation model is used for establishing a mapping relation between the electromyographic signals and the joint angles of the complex task, a good result is often not obtained, the robustness of the model is poor, and accurate estimation of the whole task cannot be smoothly completed.
Therefore, it is necessary to develop a new complex motion continuous estimation method to successfully complete the precise estimation of complex task.
Disclosure of Invention
Aiming at the defects and improvement requirements of the prior art, the invention aims to provide a complex motion continuous estimation method based on electromyography mapping model switching, and aims to solve the technical problem that complex tasks cannot be continuously estimated in the prior art.
In order to achieve the above object, the present invention provides a complex motion continuous estimation method based on electromyography model switching, which includes the following steps:
s1: collecting joint angle signals and multi-channel myoelectric signals according to the joint angle to be estimated and surface muscles driving corresponding joints to move, dividing the task into a plurality of subtasks according to the characteristics of the complex task, combining the plurality of subtasks into the whole complex task,
s2: respectively preprocessing the collected electromyographic signals and the joint angle signals,
s3: utilizing the preprocessed electromyographic signals and joint angle signals, adopting an LSTM deep learning network to establish a mapping sub-relation of the electromyographic signals and the joint angle signals to obtain an estimation sub-model,
s4: establishing a mapping relation between the electromyographic signals and each estimation submodel by using an RF ensemble learning algorithm to obtain an RF switching model,
s5: and classifying the electromyographic signal characteristics by using an RF (radio frequency) switching model, judging an estimation submodel to which the current movement belongs, and inputting the electromyographic characteristics into the corresponding estimation submodel to obtain the joint angle estimation value at the moment.
Further, in step S2, preprocessing the electromyographic signals includes filtering, feature extraction, and normalization of the electromyographic signals, specifically, performing notch filtering on the electromyographic signals to eliminate power frequency interference, performing 20Hz to 460Hz band-pass filtering on the electromyographic signals to eliminate low frequency noise and high frequency noise, performing feature extraction on the electromyographic signals subjected to noise reduction, taking feature values of the electromyographic signals by using a sliding window method, and then normalizing the electromyographic features of the channels according to the maximum value and the minimum value of the feature values of the channels to obtain the electromyographic features after normalization of the channels.
Further, in step S2, the preprocessing of the joint angle signal means that the actually measured joint angle signal and the estimated joint angle signal are respectively filtered, so that the actually measured joint angle and the estimated joint angle are input and output to the corresponding model at a relatively smooth angle.
Further, in step S3, the obtained estimation submodel is an LSTM estimation submodel, and the specific process of obtaining the LSTM estimation submodel is as follows: first, initialize the LSTM deep learning networkParameter of (2), weight matrix W of candidate state C And bias b C Weight matrix W of input gates i And bias b i Weight matrix W of forgetting gate f And bias b f Weight matrix W of output gates o And bias b o Initializing to a random number between 0 and 1, setting the neuron number of an input layer of the LSTM deep learning network to be M, setting the layer number to be i, taking the output of each layer as the input of the next layer, training the LSTM network after the initialization of the LSTM network is completed, inputting the preprocessed electromyographic signals and joint angle signals into the LSTM model, and optimizing the multiple weight matrixes and the multiple biases by adopting a back propagation algorithm to obtain the LSTM estimation sub-model.
Further, in step S5, a mapping relationship is established between the electromyographic signals and each LSTM estimation submodel by using an RF ensemble learning algorithm to obtain an RF switching model, which specifically includes: and training to obtain an RF switching model by taking the preprocessed electromyographic signals and the corresponding label of the LSTM model as input, wherein the output of the RF switching model is the label of the LSTM estimation sub-model, the label of the LSTM model is a number of 1-n, and n LSTM estimation sub-models are generated.
Furthermore, the joints in the joint angle signals are human body compound joints such as joints of human hands, shoulder and elbow joints and the like, the multichannel electromyographic signals are electromyographic signals generated by relevant surface muscles driving the compound joints to move, and the selection of the number of the channels of the electromyographic signals is determined according to a compound joint angle target estimated according to needs.
Further, the method also comprises a step S6, wherein the step S6 is as follows: and adopting the Pearson correlation coefficient and the root mean square error as evaluation indexes of the actually measured joint angle and the estimated joint angle so as to evaluate the estimation precision of the estimated joint angle.
Generally, compared with the prior art, the above technical solution conceived by the present invention has the following beneficial effects:
the invention improves the estimation precision of a single estimation submodel by utilizing the advantage of strong long-term memory of the LSTM deep learning network in the electromyographic signals. By adopting a model switching mechanism, the defect that a single model cannot be accurately estimated in a complex task is overcome, and accurate estimation under the complex task is realized. Experimental results show that the method effectively improves the estimation precision of the angle of each joint under the complex task.
Drawings
FIG. 1 is a flow chart of the implementation of the complex motion continuous estimation method based on electromyography mapping model switching;
FIG. 2 is a schematic diagram of an LSTM deep learning network employed by the present invention;
FIG. 3 is a schematic diagram of an RF handoff model employed by the present invention;
FIG. 4 is a graph of the estimation accuracy of the LSTM submodel of the present invention applied to five fingers;
FIG. 5 is a graph of the comparison of the estimation accuracy of the present invention using the LSTM hybrid estimation model in combination with the model switching mechanism with a single LSTM submodel for complex gripping tasks.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention relates to a composite motion continuous estimation method based on an electromyographic signal combined model switching mechanism and an LSTM deep learning network under a complex task. And providing an LSTM hybrid estimation model combined with a model switching mechanism to realize the continuous estimation of the human joint compound motion under complex tasks. The LSTM has great advantages in processing time series problems, introduces the cell state to store long-term memory, and improves the defect that the RNN recurrent neural network can only store short-term memory. The method has the advantages that the mapping model between the electromyographic signals and human body joint motions is built by using the LSTM deep learning network, and the estimation accuracy can be improved compared with other machine learning methods.
Specifically, for different task scenarios in real life, the task is divided into a plurality of subtasks according to task complexity characteristics, for example, the tasks are decomposed into n tasks, and different subtasks are trained by using an LSTM deep learning network respectively to obtain a plurality of LSTM estimation submodels. And training the electromyographic signals and the LSTM estimation submodels to which the electromyographic signals belong by using a Random Forest (RF) ensemble learning algorithm to obtain a switching model.
The technical scheme adopted by the invention comprises the following steps:
s1: and collecting joint angle signals and multi-channel electromyographic signals according to the joint angle to be estimated and surface muscles driving corresponding joint motion. And dividing the task into a plurality of subtasks according to the characteristics of the complex task, and combining the plurality of subtasks into the whole complex task. And selecting the human joint angles with multiple degrees of freedom as an estimation target according to the requirements of n subtasks decomposed from the complex task, and acquiring a multi-channel electromyographic signal as system input.
S2: and preprocessing the electromyographic signals, including filtering, feature extraction and normalization of the electromyographic signals. Specifically, the surface electromyogram signal is too weak and is easily interfered by a surrounding electromagnetic field, the surface electromyogram signal is easily interfered by power frequencies of joint angle measuring equipment and other electrical equipment in the process of actually measuring the electromyogram signal, and the interference frequency distribution is 50Hz and integer times of the frequency. Therefore, the electromyographic signals need to be processed for eliminating power frequency interference. Band-pass filtering processing is carried out according to the main frequency of the electromyographic signals so as to eliminate motion artifact noise (low frequency) and other high-frequency noise. Feature extraction is carried out on the electromyographic signals subjected to noise reduction processing, and different single features or combined features such as Root Mean Square (RMS), waveform Length (WL), absolute value Integral Average Value (IAV), mean Absolute Value (MAV), zero crossing coefficient (ZC), slope Sign Change (SSC), average amplitude change rate (AAC) and the like are selected according to different tasks. And respectively normalizing the electromyographic characteristics of each channel according to the maximum value and the minimum value of the characteristic value of each channel to obtain the electromyographic characteristics after normalization of each channel as input signals of an RF switching model and an LSTM model.
And preprocessing the joint angle signal, and filtering the actually measured joint angle signal and the estimated joint angle signal to input and output the joint angle to the model at a relatively smooth angle.
S3: and respectively training the LSTM estimation sub-models by taking the electromyographic features of each subtask and the corresponding joint angle signals as inputs.
Initializing parameters of the LSTM deep learning network: weighting matrix W of candidate state C And bias b C Weight matrix W of input gates i And bias b i Forgetting gate weight matrix W f And bias b f Weight matrix W of output gates o And bias b o Initialized to random numbers between 0 and 1. And setting the number of neurons in an input layer of the LSTM deep learning network as M, setting the number of layers as i, and using the output of each layer as the input of the next layer.
And training the LSTM network after the initialization of the LSTM network is completed, inputting the preprocessed electromyographic signals and joint angle signals into the LSTM model, and optimizing the weight matrix and the bias by adopting a back propagation algorithm to obtain an LSTM estimation sub-model. The LSTM model is also the LSTM network.
Setting the number of training iterations as epoch, loss function as loss, optimizer as optimizer, and batch size as the number of samples selected in one training.
S4: and training an RF switching model, namely taking the preprocessed electromyographic signals and the corresponding label of the LSTM model as model input to train the RF switching model, wherein the output of the RF switching model is the label of the LSTM estimation sub-model. Performing bootstrap sampling on input samples to obtain k sample sets, respectively training the k sample sets by using a decision tree model to obtain k tree classifiers, wherein labels of the LSTM models are numbers 1-n, n LSTM estimation submodels are generated in total, the maximum depth of each tree classifier is set to be m, a gini function calculation method is adopted for judging whether nodes are continuously split, the minimum sample number required by node splitting is set to be f, the maximum leaf node number is not limited, and the maximum feature number participating in judgment during node splitting is equal to all feature numbers.
S5: after the RF switching model and each LSTM estimation sub-model are trained, the whole LSTM hybrid estimation model combined with the model switching mechanism is trained, and the hybrid model is used for carrying out motion continuous estimation on newly input electromyographic signal data in a test stage under a complex task. Particularly, in the estimation stage, the electromyographic features of all channels are normalized by using the maximum value and the minimum value of the electromyographic features of all channels in the training stage, the normalized electromyographic features are input into the RF switching model, the LSTM estimation submodel to which the corresponding electromyographic signals belong is output, then the normalized electromyographic features are input into the corresponding LSTM estimation submodel for estimation, and the estimation value of each joint angle is output.
In practical engineering practice, the composite motion continuous estimation effect of the LSTM hybrid estimation model combined with the model switching mechanism provided by the invention under a complex task is evaluated, and during evaluation, firstly, the estimation accuracy of a single LSTM model in each estimation sub-model is evaluated to verify the estimation accuracy of each estimation sub-model, and secondly, the estimation effect comparison of the single LSTM model and the LSTM hybrid estimation model combined with the model switching mechanism under the complex task is compared to verify whether the LSTM hybrid estimation model combined with the model switching mechanism can better adapt to the change of a task scene or not, so that the estimation of the complex task has higher estimation accuracy and robustness. The evaluation indexes are Pearson Correlation Coefficient (CC) and Root Mean Square Error (RMSE). The result shows that the method can effectively improve the estimation precision of the angle of each joint under the complex task.
The whole flow chart during estimation is shown in fig. 1, electromyographic signals are preprocessed to obtain electromyographic characteristics, the electromyographic characteristics are input into an RF (radio frequency) switching model to obtain classification results, labels of an LSTM estimation sub-model are output, and the electromyographic characteristics enter the corresponding LSTM estimation sub-model to output an estimated compound motion angle. Because the results need to be compared, the characteristic value obtained after the electromyographic signals are preprocessed can also be directly output by the composite motion angle through a single LSTM model. The angle outputs respectively obtained by the two models can be compared in CC and RMSE dimensions, so that the good estimation effect of the LSTM hybrid estimation model combined with the model switching mechanism under a complex task is highlighted.
The invention is described in further detail below with reference to the figures and the specific embodiments. In particular, the complicated task involved in the embodiments is that people need to grip different objects with different forces in daily life, for example, when gripping a paper cup, a small force is needed to ensure the gripping, when gripping a pot handle to pick up a cooking pot, a large force is needed, and in ordinary life, only a moderate force is needed to take a bottle of water.
(1) And dividing the task into a plurality of subtasks according to the characteristics of the complex task, and combining the plurality of subtasks into the whole complex task. According to the characteristic that people can grasp different objects by adopting different forces in the daily life situation, the joint angle of the hand with five degrees of freedom is selected as an estimation target, and the six-channel electromyographic signal is selected as system input. The five-freedom-degree hand joints are respectively a thumb metacarpophalangeal joint, an index finger metacarpophalangeal joint, a middle finger metacarpophalangeal joint, a ring finger metacarpophalangeal joint and a little finger metacarpophalangeal joint. The six-channel muscle signals are the flexor hallucis longus, the flexor hallucis digitalis, the flexor digitorum profundus, the extensor digitorum, and the extensor hallucis longus, respectively. And collecting joint angle signals and multi-channel myoelectric signals according to the joint angle to be estimated and surface muscles driving the corresponding joints to move.
(2) And preprocessing the electromyographic signals, including filtering, feature extraction and normalization of the electromyographic signals.
(2a) Filtering: because the surface electromyogram signal is too weak and is easily interfered by surrounding electromagnetic fields, the surface electromyogram signal is interfered by power frequency of finger angle measuring equipment and other electrical equipment in the process of actually measuring the electromyogram signal, and the interference frequency is distributed at 50Hz and integral multiples thereof. Therefore, a notch filtering process (Comb-notch Filter) is performed on the electromyographic signals to eliminate power frequency interference. As the main frequency of the electromyographic signals is concentrated at 20-460 Hz, the electromyographic signals are subjected to band-pass filtering processing at 20-460 Hz to eliminate motion artifact noise (low frequency) and other high-frequency noise.
(2b) Feature extraction is carried out on the electromyographic signals subjected to noise reduction processing, the main variable factor in the complex situation task is force, and the average absolute value feature of the electromyographic signals represents the activity level of muscles so as to reflect the gripping force to a certain extent, and therefore the average absolute value (MAV) is selected as the feature. The electromyographic signals are subjected to characteristic value acquisition by adopting a sliding window method, the size of the sliding window is set to be 200ms, the sliding window is backwards slid for 50ms each time, and 75% of overlapped data exists between adjacent windows.
(2c) And after the electromyographic characteristics are obtained, respectively normalizing the electromyographic characteristics of each channel according to the maximum value and the minimum value of the characteristic value of each channel, and taking the electromyographic characteristics after normalization of each channel as input signals of a subsequent RF switching model and an LSTM model (two subsequent models).
Preprocessing a five-degree-of-freedom finger metacarpophalangeal joint angle signal, and performing Butterworth low-pass filtering processing (cutoff frequency 2Hz, 4-order) on the actually measured angle signal to enable the joint angle to be input into an LSTM estimation model at a relatively smooth angle.
(3) Dividing the task into 3 subtasks according to the task complexity, wherein the subtasks are respectively a task one: grip the object with less force, task two: gripping an object with moderate force, task three: the object is gripped with a large force. And respectively training LSTM estimation submodels by taking the electromyographic characteristics of each subtask and the corresponding joint angle signals as inputs, namely training three LSTM estimation submodels corresponding to three different levels of force. A schematic diagram of a principle of a single LSTM deep learning network is shown in FIG. 2, when each model is trained, the preprocessed myoelectric signals and the preprocessed joint angle signals are input into an input gate, a forgetting gate, an output gate and candidate states in the LSTM deep learning network, and output values of the candidate states of the LSTM deep learning network at t moment are obtained
Output value i of input gate t Output value f of forgetting gate t And the output value o of the output gate t Wherein:
i t ＝σ(W i ·[h t-1 ,x t ]+b i )
f t ＝σ(W f ·[h t-1 ,x t ]+b f )
o t ＝σ(W o ·[h t-1 ,x t ]+b o )
h t ＝o t *tanh(C t )
wherein x is t The electromyographic signals after being preprocessed represent the input of the model. W C And b C Weight matrix and offset, W, representing candidate states, respectively i And b i Respectively representing the weight matrix and the offset, W, of the input gate f And b f Weight matrix and offset, W, representing forgetting gates, respectively o And b o Weight matrix and offset, C, representing the output gates, respectively t 、C t-1 Respectively representing the output of the cell state of the storage long-term memory at t and t-1,
respectively representing the output of the candidate state at times t, t-1, h t 、h t-1 Respectively representing the output of the LSTM deep learning network at t and t-1, wherein sigma (DEG) represents a sigmoid activation function, and tanh (DEG) represents a tanh activation function.
Initializing parameters of the LSTM deep learning network: weighting matrix W of candidate state C And bias b C Weight matrix W of input gates i And bias b i Forgetting gate weight matrix W f And bias b f Weight matrix W of output gates o And bias b o Initialized to random numbers between 0 and 1. And setting the number of neurons in an input layer of the LSTM deep learning network as M, setting the number of layers as i, and using the output of each layer as the input of the next layer.
Training the LSTM network after the initialization of the LSTM network is completed: inputting the preprocessed electromyographic signals and joint angle signals into an LSTM model, optimizing the weight matrix and the bias by adopting a back propagation algorithm to obtain an LSTM training model, and performing the same operation on the three LSTM models to obtain three different LSTM estimation submodels. Setting the number of training iterations as epoch, the loss function as loss, the optimizer as optimizer, and the number of samples selected in one training as batch size. In this example, M =256, i =2, epoch =150, loss = mae (mean absolute error), optimizer = Adam (Adam optimizer), and batch size =16.
(4) And (3) training an RF switching model, namely taking the preprocessed electromyographic signals and the corresponding labels of the LSTM model as model inputs to train the RF switching model, and taking the output of the RF switching model as the labels of the LSTM estimation sub-model. The schematic diagram of the RF switching model is shown in fig. 3, bootstrap sampling is performed on input samples to obtain k sample sets, the k sample sets are trained by using a decision tree model to obtain k tree classifiers, wherein the LSTM submodels are labeled with numbers 1-n (to generate n LSTM estimation submodels), the maximum depth of each tree classifier is set to m, whether nodes are continuously split is judged by adopting a gini function calculation method, the minimum number of samples required for splitting is set to f, the maximum number of leaf nodes is not limited, and the maximum number of features participating in judgment when nodes are split is equal to all the feature numbers. In this embodiment, the sample set is divided into 10 sample sets, i.e. k =10, according to bootstrap sampling, there are three LSTM estimation submodels n =3, the maximum depth of each tree classifier is set to 2, i.e. m =2, and the minimum number of samples required for splitting is set to 2, i.e. f =2.
(5) After the RF switching model and each LSTM estimation sub-model are trained, the LSTM hybrid estimation model combined with the model switching mechanism is trained, and the model is used for carrying out composite motion continuous estimation on newly input electromyographic signal data under a complex task. In the estimation stage, the electromyographic features of all channels are normalized by using the maximum value and minimum value of the electromyographic features of all channels in the training stage, the normalized electromyographic features are input into an RF switching model, an LSTM estimation sub-model to which the corresponding electromyographic features belong is output, then the normalized electromyographic features are input into the corresponding LSTM estimation sub-model for estimation, and the estimation value of the angle of each joint is output.
The myoelectric compound motion continuous estimation effect of the LSTM mixed estimation model of the set model switching mechanism provided by the invention under a complex task is evaluated.
And evaluating the estimation accuracy of the single LSTM model in each estimation submodel by using the Pearson Correlation Coefficient (CC) and the Root Mean Square Error (RMSE). The estimation accuracy of the LSTM deep learning network is illustrated below by a single LSTM deep learning network constructed in the moderate force context of the present embodiment: average CC =0.944 and average RMSE =11.63 °. CC >0.8 indicates a strong correlation between the estimated angle and the measured angle, and RMSE indicates a root mean square error of 11.63 ° for the five finger angle, as shown in fig. 4.
Compared with the estimation effect of the single LSTM model and the LSTM hybrid estimation model combined with the model switching mechanism under the complex task, the result shows that the single LSTM deep learning network cannot adapt to the accurate estimation under the complex task, and the LSTM hybrid estimation model combined with the model switching mechanism can adapt to the scene change to realize the accurate estimation under the complex task. The estimation effect of the single LSTM and the LSTM hybrid estimation model combined with the model switching mechanism under the complex task of force variation in the embodiment is further described below.
The method comprises the steps of respectively carrying out cross estimation on the grasping motion under different force grade conditions by using a single LSTM submodel obtained by training under different force grade conditions, and comparing the cross estimation with the continuous estimation on the grasping motion under different force grade conditions by using an LSTM mixed estimation model combined with a model switching mechanism, wherein the estimation effects of each model are as shown in the following table I and table II, wherein L, M and H models respectively represent LSTM submodels obtained by training with low-grade force, medium-grade force and high-grade force, and an S model represents an LSTM mixed estimation model combined with the model switching mechanism. The L, M and H forces respectively represent low-level force, medium-level force and high-level force input in estimation. The evaluation index of the pair of estimation results in the table is a Pearson Correlation Coefficient (CC), and the evaluation index of the pair of estimation results in the table is a Root Mean Square Error (RMSE).
Table i evaluation index of estimation result is the result of pearson correlation coefficient
The evaluation index of the second estimation result is the result of the root mean square error
The first table and the second table show the estimation effect of each model on the forces of different levels, and it can be seen from the tables that the single LSTM model under each force level condition only has a good estimation effect on the force level condition under the model, and the estimation effect on the force conditions of other levels is poor. The LSTM hybrid estimation model combined with the model switching mechanism has a good estimation effect on different level force conditions, and a comparison result shows that the LSTM hybrid estimation model combined with the model switching mechanism has better robustness and estimation accuracy on complex tasks. A comparison graph of the estimation effect of the estimation model obtained by training under the low-level force condition and the estimation effect of the LSTM hybrid estimation model combined with the model switching mechanism on the grip under the three force level conditions is shown in fig. 5. As can be seen from fig. 5, the angle curve estimated by the LSTM hybrid estimation model combined with the model switching mechanism under the condition of mixing the three force levels fits the actually measured angle curve well, while the angle curve estimated by the single LSTM estimation model trained under the low-level force condition does not fit the actually measured angle curve seriously under the conditions of medium-level force and high-level force. Therefore, a single LSTM deep learning network cannot adapt to accurate estimation under a complex task, and an LSTM hybrid estimation model combined with a model switching mechanism can adapt to scene changes to realize accurate estimation under the complex task.
The method can automatically switch under different scenes, and accurately estimate by using the LSTM estimation sub-model after determining the scenes, thereby ensuring the algorithm robustness under complex tasks and realizing the continuous estimation of the compound motion under the complex tasks based on the electromyographic signals.
In the invention, in order to accurately distinguish the training stage, the estimation stage and different estimation models, an LSTM deep learning network, an LSTM model and an LSTM submodel are used. Before an accurate estimation model is obtained, namely before all parameters of the model are trained, an LSTM deep learning network is used, the LSTM model refers to an estimation model after all parameters in the LSTM deep learning network are trained, an LSTM submodel refers to estimation models respectively corresponding to different scenes in the same complex task, the LSTM submodel refers to an estimation model which accurately identifies myoelectric characteristics as a certain scene in the complex task by using an RF integrated learning algorithm, and the LSTM is an acronym of longShort-TermMemory. RF is an acronym for Random Forest, representing a Random Forest classifier.
It will be understood by those skilled in the art that the foregoing is only an exemplary embodiment of the present invention, and is not intended to limit the invention to the particular forms disclosed, since various modifications, substitutions and improvements within the spirit and scope of the invention are possible and within the scope of the appended claims.
Claims (6)
1. A complex motion continuous estimation method based on electromyography model switching is characterized by comprising the following steps:
s1: collecting joint angle signals and multi-channel electromyographic signals according to the joint angle to be estimated and surface muscles driving corresponding joints to move, dividing a task into a plurality of subtasks according to the characteristics of a complex task, combining the plurality of subtasks into the whole complex task,
s2: respectively preprocessing the collected electromyographic signals and joint angle signals,
s3: utilizing the preprocessed electromyographic signals and joint angle signals, adopting an LSTM deep learning network to establish a mapping sub-relation of the electromyographic signals and the joint angle signals to obtain an estimation sub-model,
s4: establishing a mapping relation between the electromyographic signals and each estimation submodel by using an RF ensemble learning algorithm to obtain an RF switching model,
s5: classifying the electromyographic signal characteristics by using an RF switching model, judging an estimation submodel to which the current movement belongs, inputting the electromyographic characteristics into the corresponding estimation submodel to obtain a joint angle estimation value at the moment,
the method comprises the following steps of establishing a mapping relation between an electromyographic signal and each LSTM estimation submodel by using an RF ensemble learning algorithm to obtain an RF switching model, and specifically comprises the following steps: and taking the preprocessed electromyographic signals and the corresponding labels of the LSTM models as input, training to obtain an RF switching model, wherein the output of the RF switching model is the label of the LSTM estimation sub-model, the labels of the LSTM models are numbers 1-n, and n LSTM estimation sub-models are formed jointly.
2. The method of claim 1, wherein the step S2 of pre-processing the electromyographic signals comprises filtering, feature extraction and normalization of the electromyographic signals,
in particular, the electromyographic signals are subjected to notch filtering processing to eliminate power frequency interference, band-pass filtering processing of 20 Hz-460 Hz is also carried out on the electromyographic signals to eliminate low-frequency noise and high-frequency noise,
extracting the feature of the electromyographic signal after noise reduction, adopting a sliding window method to extract the feature value of the electromyographic signal,
and after the electromyographic features are obtained, normalizing the electromyographic features of the channels according to the maximum value and the minimum value of the characteristic values of the channels respectively to obtain the electromyographic features after normalization of the channels.
3. The method as claimed in claim 2, wherein the step S2 of preprocessing the joint angle signal is to filter the measured joint angle signal and the estimated joint angle signal, respectively, so that the measured joint angle and the estimated joint angle are input and output to the corresponding model at a relatively smooth angle.
4. The method for continuous estimation of complex motion based on electromyography model switching as claimed in claim 3, wherein the estimation submodel obtained in step S3 is an LSTM estimation submodel, and the specific process for obtaining the LSTM estimation submodel is as follows:
firstly, initializing parameters of the LSTM deep learning network, and converting the weight matrix W of the candidate state C And bias b C Weight matrix W of input gates i And bias b i Weight matrix W of forgetting gate f And bias b f Weight matrix W of output gates o And bias b o Initializing to random number between 0 and 1, setting the neuron number of the input layer of the LSTM deep learning network as M, setting the layer number as i, using the output of each layer as the input of the next layer,
after the initialization of the LSTM deep learning network is completed, the LSTM deep learning network is trained, the preprocessed electromyographic signals and the preprocessed joint angle signals are input into the LSTM deep learning network, and the plurality of weight matrixes and the plurality of biases are optimized by adopting a back propagation algorithm to obtain an LSTM estimation sub-model.
5. The method for continuously estimating complex motion based on electromyography model switching as claimed in claim 4, wherein the joints in the joint angle signals are human body compound joints including joints of human hands and joints of shoulders and elbows, and the multichannel electromyography signals are electromyography signals generated by relevant surface muscles driving the compound joints to move.
6. The method for continuous estimation of complex motion based on electromyography model switching according to claim 5, further comprising step S6, wherein step S6 is: and adopting the Pearson correlation coefficient and the root mean square error as evaluation indexes of the actually measured joint angle and the estimated joint angle so as to evaluate the estimation precision of the estimated joint angle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011066204.7A CN112232161B (en) | 2020-09-30 | 2020-09-30 | Complex motion continuous estimation method based on electromyography mapping model switching |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011066204.7A CN112232161B (en) | 2020-09-30 | 2020-09-30 | Complex motion continuous estimation method based on electromyography mapping model switching |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112232161A CN112232161A (en) | 2021-01-15 |
| CN112232161B true CN112232161B (en) | 2023-04-07 |
Family
ID=74119337
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011066204.7A Active CN112232161B (en) | 2020-09-30 | 2020-09-30 | Complex motion continuous estimation method based on electromyography mapping model switching |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112232161B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114055461B (en) * | 2021-08-20 | 2023-02-14 | 华中科技大学 | Robot force and position synchronous teleoperation control method and device based on myoelectric interface |
| CN114733160B (en) * | 2022-04-14 | 2022-10-18 | 福州大学 | Myoelectric signal-based muscle strength training equipment control method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108874149A (en) * | 2018-07-28 | 2018-11-23 | 华中科技大学 | A method of continuously estimating human synovial angle based on surface electromyogram signal |
| CN109171707A (en) * | 2018-10-24 | 2019-01-11 | 杭州电子科技大学 | A kind of intelligent cardiac figure classification method |
| CN109262618A (en) * | 2018-12-12 | 2019-01-25 | 武汉理工大学 | Upper limb multi-joint synchronous proportional myoelectric control method and system based on muscle collaboration |
| CN109833031A (en) * | 2019-03-12 | 2019-06-04 | 西安交通大学 | It is a kind of that the sleep mode automatically method by stages of more physiological signals is utilized based on LSTM |
| CN110464348A (en) * | 2019-07-10 | 2019-11-19 | 深圳市智能机器人研究院 | The continuous amount of exercise recognition methods of joint of lower extremity and system based on electromyography signal |
| CN111079590A (en) * | 2019-12-04 | 2020-04-28 | 东北大学 | Peripheral vehicle behavior pre-judging method of unmanned vehicle |
| CN111134666A (en) * | 2020-01-09 | 2020-05-12 | 中国科学院软件研究所 | Emotion recognition method of multi-channel electroencephalogram data and electronic device |
-
2020
- 2020-09-30 CN CN202011066204.7A patent/CN112232161B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108874149A (en) * | 2018-07-28 | 2018-11-23 | 华中科技大学 | A method of continuously estimating human synovial angle based on surface electromyogram signal |
| CN109171707A (en) * | 2018-10-24 | 2019-01-11 | 杭州电子科技大学 | A kind of intelligent cardiac figure classification method |
| CN109262618A (en) * | 2018-12-12 | 2019-01-25 | 武汉理工大学 | Upper limb multi-joint synchronous proportional myoelectric control method and system based on muscle collaboration |
| CN109833031A (en) * | 2019-03-12 | 2019-06-04 | 西安交通大学 | It is a kind of that the sleep mode automatically method by stages of more physiological signals is utilized based on LSTM |
| CN110464348A (en) * | 2019-07-10 | 2019-11-19 | 深圳市智能机器人研究院 | The continuous amount of exercise recognition methods of joint of lower extremity and system based on electromyography signal |
| CN111079590A (en) * | 2019-12-04 | 2020-04-28 | 东北大学 | Peripheral vehicle behavior pre-judging method of unmanned vehicle |
| CN111134666A (en) * | 2020-01-09 | 2020-05-12 | 中国科学院软件研究所 | Emotion recognition method of multi-channel electroencephalogram data and electronic device |
Non-Patent Citations (1)
| Title |
|---|
| "Simultaneous and Proportional Estimation of Multijoint Kinematics From EMG Signals for Myocontrol of Robotic Hands";Qin Zhang et al;《IEEE/ASME TRANSACTIONS ON MECHATRONICS》;20200831;第1953-1960页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112232161A (en) | 2021-01-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bao et al. | A CNN-LSTM hybrid model for wrist kinematics estimation using surface electromyography | |
| Allard et al. | A convolutional neural network for robotic arm guidance using sEMG based frequency-features | |
| CN112232161B (en) | Complex motion continuous estimation method based on electromyography mapping model switching | |
| CN101564328B (en) | Laptop artificial limb multi-movement-mode identifying method based on support vector data description | |
| CN107618018B (en) | Manipulator action speed proportional control method based on myoelectricity | |
| CN108874149B (en) | Method for continuously estimating human body joint angle based on surface electromyogram signal | |
| CN101859377A (en) | Electromyographic signal classification method based on multi-kernel support vector machine | |
| CN110399846A (en) | A kind of gesture identification method based on multichannel electromyography signal correlation | |
| CN113059570B (en) | Human-robot cooperative control method based on human body dynamic arm strength estimation model | |
| CN110826625A (en) | Finger gesture classification method based on surface electromyographic signals | |
| Zhao et al. | A five-fingered underactuated prosthetic hand control scheme | |
| Godoy et al. | Electromyography-based, robust hand motion classification employing temporal multi-channel vision transformers | |
| Nazemi et al. | Artificial neural network classifier in comparison with LDA and LS-SVM classifiers to recognize 52 hand postures and movements | |
| Liang et al. | Research on recognition of nine kinds of fine gestures based on adaptive AdaBoost algorithm and multi-feature combination | |
| Zhao et al. | EMG control for a five-fingered prosthetic hand based on wavelet transform and autoregressive model | |
| Lin et al. | A normalisation approach improves the performance of inter-subject sEMG-based hand gesture recognition with a ConvNet | |
| CN113171214A (en) | Multi-path feedback myoelectricity control prosthetic hand based on self-adaptive enhanced classifier and method | |
| Kan et al. | EMG pattern recognition based on particle swarm optimization and recurrent neural network | |
| CN111985270B (en) | sEMG signal optimal channel selection method based on gradient lifting tree | |
| Taghizadeh et al. | Classification of Electromyography Signals Using Neural Networks and Features From Various Domains | |
| CN106709934A (en) | Frequency domain Gaussian kernel function image tracking method | |
| CN115512440A (en) | Cross-user gesture recognition method based on multi-source domain self-adaption | |
| CN113729738B (en) | Construction method of multichannel myoelectricity characteristic image | |
| CN113951904A (en) | sEMG-based method for continuously predicting knee joint angle of lower limb | |
| CN114343679A (en) | Surface electromyogram signal upper limb action recognition method and system based on transfer learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |